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1.1  AREAS  OF  INTEREST 

•  Enabling  the  DoD  vision  of  achieving  training  transformation. 

•  Fulfilling  the  Air  Force’s  Distributed  Mission  Operations  vision  by  providing  capabilities  to  train  airmen 
to  meet  Combatant  Commanders’  requirements. 

1.2  CURRENT  RESEARCH  PROJECTS 

1.2.1  Distributed  Mission  Training  (DMT) 

DMT  is  a  shared  training  environment  comprised  of  live,  virtual,  and  constructive  simulations  allowing 
warfighters  to  train  individually  or  collectively  at  all  levels  of  war.  DMT  allows  multiple  players  at  multiple 
sites  to  engage  in  individual  and  team  participation  to  full  theater-level  battles.  It  allows  participation, 
using  almost  any  type  of  networkable  training  device,  from  each  weapon  system  and  mission  area. 
Additionally,  computer-generated,  or  constructive,  forces  can  be  used  to  substantially  enhance  the  scenario. 
This  combination  of  live,  virtual,  and  constructive  environments  allows  nearly  unlimited  training 
opportunities  for  joint  and  combined  forces  from  their  own  location  or  a  deployed  training  site. 

1.2.2  DMT  Air 

DMT  for  aircrew  complements  flying  training  by  providing  capabilities  to  gain  knowledge,  skills,  and 
experience  for  infrequently  practiced  tasks  and  missions.  Since  DMT  is  not  constrained  by  airspace  limitations 
or  by  number  of  aircraft  available,  pilots  and  controllers  can  learn  to  conduct  complex  missions  in  high-threat 
enviromnents  focusing  on  enhancing  the  individual  and  team  skills  required  for  mission  success. 

Training  Systems  Technology  Research:  USAF  F-16  pilots  and  AW  ACS  controllers  participate  in  training 
research  exercises  using  the  F-16  DMT  testbed  at  AFRL/HEA  in  Mesa,  AZ.  Pilots  ranging  in  experience  from 
Mission  Qualification  through  Weapons  School  use  the  testbed  to  enhance  their  skills.  Research  is  focused  on 
developing  training  strategies  that  emphasize  specific  training  needs  and  include  validated  measures  of 
performance  to  track  progress.  In  addition,  coalition  training  research  exercises  are  conducted  using  real-time, 
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secure  links  to  defence  laboratories  in  Canada  and  the  UK.  US,  Canadian,  and  British  warfighters  use  DMT  to 
plan,  brief,  fly,  and  debrief  simulated  composite  force,  coalition  missions. 

High-Resolution  Visuals:  The  full  field-of-view  visual  displays  currently  used  in  the  F-16  DMT  testbed 
provide  high  levels  of  immersion  into  the  synthetic  environment  but  lack  the  resolution  required  for  within- 
visual-range  air  combat  and  close  air  support  missions.  Research  is  in  progress  to  increase  resolution  in 
immersive  displays  to  eye  limiting  (20/20  visual  acuity)  resolution.  The  objective  of  the  20/20  Immersive 
Visual  Display  System  for  the  DMT-Aircrew  program  is  to  develop,  integrate,  and  demonstrate  a  high  fidelity 
visual  display  system  for  flight  simulators.  This  Air  Combat  Command  (ACC)  Category  1,  Advanced 
Technology  Demonstration  will  significantly  enhance  the  visual  system  for  fighter  simulators  such  as  the  F-16 
by  developing  and  integrating  technologies  that  will  improve  the  resolution  of  the  simulated  visual  and  sensor 
imagery  while  reducing  the  overall  cost  of  the  image  generation  and  display  system.  This  display  system  will 
provide  bright,  high-resolution  (20/20  acuity)  imagery  wherever  the  pilot  looks.  The  resulting  system  will 
eliminate  the  need  for  target  projectors  that  restrict  the  number  of  high-resolution  targets  that  can  be  displayed 
or  channel  the  pilot’s  attention  to  specific  areas  within  the  scene.  In  addition,  this  display  system  will  improve 
the  pilot’s  perception  of  size  and  distance  over  real-image  displays.  This  Advanced  Technology 
Demonstration  is  scheduled  for  completion  in  FY  05. 

The  approach  for  achieving  the  objective  is  through  contracted  efforts  to  design,  develop,  and  deliver: 

•  High  resolution,  full  color  laser  projectors  capable  of  displaying  over  10  times  the  number  of  pixels 
currently  displayed  by  high  resolution  projectors;  and 

•  High  performance,  low  cost  image  generators  based  on  commercial  technology  that  can  provide  high 
resolution  visual  and  sensor  imagery  at  60  HZ. 

The  Next  Threat  System:  Constructive  (computer-generated)  entities  are  a  vital  component  of  DMT. 
Constructive  entities  serve  as  both  friendly  and  threat  aircraft  and  provide  an  Integrated  Air  Defense  System 
incoiporating  missiles,  radars,  and  a  command  network.  TNTS  is  an  object  like  electronic  warfare 
environment  system  designed  to  be  HLA  compliant.  The  design  purpose  of  TNTS  is  to  accurately  depict 
Electronic  Warfare  considerations  in  distributed  mission  training. 

Distributed  Training  Network  Guard:  Opportunities  for  team  and  inter-team  training  using  DMT  are 
limited  by  the  different  levels  of  security  required  by  different  systems.  The  Distributed  Network  Training 
Guard  will  greatly  enhance  DMT  and  Distributed  Mission  Operations  (DMO)  by  allowing  distributed 
simulations  to  interoperate  at  their  native  classification  levels.  DTNG  will  expand  the  range  of  possible 
scenarios  exploited  by  DMO,  and  further  enable  warfighters  to  “train  the  way  we  fight”  in  a  distributed, 
secure,  high-fidelity,  full-mission  training  synthetic  battlespace.  The  DTNG  is  a  Category  I  Advanced 
Technology  Demonstration  program  that  delivers  a  capability  to  transfer  data  between  High-Level 
Architecture  (HLA)-based  distributed  simulation  networks  executing  at  multiple  security  levels,  thereby 
providing  a  critical  element  in  realizing  the  full  potential  of  the  DMO  concept.  The  operating  component  of 
the  DTNG  is  a  physical,  real-time  automated  network  guard  that  supports  real-time  two-way  data  transfer 
between  HLA  simulation  federations  operating  at  different  security  levels.  The  parameters  for  this  guard  are 
set  pre-mission  via  a  standalone  interface  that  provides  the  security  and  federation  domain  experts  with  a  tool 
to  develop  and  review  reclassification  rules  that  govern  the  transfer  of  objects,  attributes,  interactions, 
parameters,  and  the  execution  of  cross  security-level,  Run-Time  Infrastructure  operations  for  cross-federation 
object  models.  These  rules  can  be  Boolean  “yes/no”  rules,  or  more  sophisticated  guising  or  sanitizing  rules. 
For  example,  a  filtering  rule  may  zero  out,  clear,  or  null  the  data  values  of  attributes,  sub-attributes, 
parameters  and  subparameters.  Guising  or  sanitizing  rules  allow  these  attribute  or  parameter  values  to  be 
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changed  within  the  constraints  of  the  data  type.  For  example,  an  F-22  operating  within  the  high-side 
federation  could  be  guised  to  appear  as  an  F-15  on  the  low-side  federation. 

Space:  DMT  is  not  limited  to  aircrew  training.  A  major  goal  of  the  DMT  -  Space  R&D  program  is  to  develop 
and  demonstrate  a  variety  of  technology  delivery  and  performance  support  options  that  will  provide  faster, 
better,  cheaper,  seamless  training  and  mission  performance  support.  Another  goal  is  to  facilitate  integration  of 
products  from  this  effort  into  our  Distributed  Mission  Operations  synthetic  battlespace  permitting  space  and 
missile  operations  that  include  the  capacity  to  provide  performance  support  and  training  to  new  operators 
within  the  context  of  mission  planning  and  rehearsal.  This  work  provides  the  scientific  and  technology 
foundation  for  future  large-scale  research  for  distributed  training  and  rehearsal  technologies  for  Space  and 
Missile  Operations.  More  specifically,  we  plan  to  collect  into  a  database  the  operational  requirements  for 
space  and  missile  operations  mission  rehearsal  and  training  as  well  as  specify  alternative  science  and 
technology  solutions  to  address  identified  needs. 

Air  and  Space  Operations  Center:  In  2000,  the  Air  and  Space  Operations  Center  (AOC)  was  designated  an 
official  Air  Force  weapon  system.  This  designation  levied  a  requirement  to  develop  a  comprehensive  training 
program  for  AOC  warfighters  from  initial  qualification  through  advanced  and  continuation  training.  The  Air 
Force  recognized  that  AOC  training  was  previously  accomplished  on-the-job  in  an  ad-hoc  fashion.  The  goal 
of  the  current  research  effort  is  to  ensure  that  AOC  training  methods  and  technologies  provide  the  operators 
with  the  training  required  to  employ  the  AOC  as  an  effective  warfighting  instrument.  AFRL/HEA  is  working 
with  ACC/DOY  and  AFC2ISRC/DO  to  gain  access  to  AOC  subject  matter  experts  to  define  the  mission 
essential  competencies  (MECs),  and  knowledge,  skills,  and  experiences  required  to  develop  an  expert  AOC 
warfighter.  MEC  definitions  are  also  needed  if  AOC  crews  are  to  be  “certified”  for  operations. 

Security  Forces:  The  objective  of  Security  Forces  DMT  research  is  to  develop,  demonstrate,  and  evaluate  a 
computer-driven,  simulation  capability  to  support  training  in  command  and  control  of  USAF  security  forces. 
The  S&T  value  resides  in  three  areas:  research  and  development  of  realistic  computer-generated  forces  to 
serve  as  stimuli  for  decision-making,  development  of  simulation  training  scenarios  to  support  learning 
objectives  underlying  mission  essential  competencies,  and  collection/analysis  of  empirical  data  to  determine 
usability  and  training  value. 

1.2.3  Night  Vision  Training  System  (NVTS) 

The  goal  of  the  NVTS  program  is  to  produce  high  fidelity,  deployable,  low-cost,  NVG  simulation  that  will 
enable  mission  training,  preview,  and  rehearsal  whenever  and  wherever  necessary.  NVTS  is  a  research  and 
development  effort  that  will  continue  development  and  transition  to  users.  The  NVG  imagery  is  based  upon 
the  modeling  of  the  unique  two-dimensional  NVG  effects  such  as  halos,  gain  response  based  on  an  accurate 
characterization  of  goggle  sensitivity,  gain,  resolution,  color,  and  field  of  view.  The  imagery  is  presented 
through  a  head  tracked  CRT-based  display  mounted  in  an  actual  NVG  shell.  This  approach  allows  for  the 
correct  eye-point  for  all  crewmembers.  Each  display  requires  at  least  one  channel  of  imagery.  The  three- 
dimensional  world  incorporates  high-resolution  material-classified  imagery,  and  accurate  per-texel 
radiometric  response  of  surface  reflectance  and  aspect.  The  current  approach  results  in  a  single  database 
which  will  support  completely  correlated  visible  and  multiple  sensor  simulation. 

1.2.4  Uninhabited  Aerial  Vehicle  Synthetic  Task  Environment 

AFRL/HEA’s  Mesa  Research  Site  has  established  a  Performance  and  Learning  Models  Lab  (PALM  Lab)  with 
the  mission  of  conducting  empirical  research  and  creating  computational  process  models  for  understanding 


RTO-TR-HFM-1 21  -Part-1 


7-5 


HUMAN  FACTORS  ISSUES  IN  THE  USE  OF  VIRTUAL 

AND  AUGMENTED  REALITY  FOR  MILITARY  PURPOSES  -  USA 


ORGANIZATION 


human  performance  and  learning.  Architectures  for  developing  computational  process  models  of  human 
behavior  in  dynamic,  time-critical  environments  have  only  recently  reached  a  state  of  maturity  that  makes 
them  useful  in  a  military  modeling  context.  Computational  process  modeling  provides  value  in  both  basic 
research  on  human  behavior  representation  and  processes,  and  also  in  applied  research  on  technologies  for 
improving  warfighter  training.  Broadly  stated,  our  objectives  in  the  current  PALM  Lab  basic  research 
program  are  to  (1)  advance  the  state  of  the  art  in  computational  process  modeling  of  human-system  interaction 
in  dynamic,  time-critical  environments  of  relevance  to  the  warfighter,  and  (2)  transition  products  and  lessons 
learned  both  to  the  scientific  community  and  to  applied  research  involving  warfighter  modeling.  In  pursuing 
these  objectives,  our  first  goal  is  to  develop  a  computational  process  model  representing  the  behavior  of  an 
Uninhabited  Air  Vehicle  (UAV)  operator.  This  model  is  being  developed  with  the  Atomic  Components  of 
Thought-Rational  (ACT-R)  human  behavior  modeling  architecture.  ACT-R  has  evolved  into  an  embodied 
cognitive  architecture  that  provides  theory-based  mechanisms  for  representing  perceptual  inputs,  cognitive 
processes,  and  motor  actions,  making  it  useful  for  modeling  interactive  tasks  like  operating  a  UAV. 

In  summary,  PALM  Lab  researchers  are  using  and  advancing  the  state  of  the  art  in  human  behavior 
representation.  Results  of  these  efforts  will  lead  to  an  improved  scientific  understanding  of  modeling  idioms 
and  representational  assumptions  useful  in  accounting  for  behavior  in  complex,  dynamic,  time-critical 
domains. 

1.3  RESULTS  ACHIEVED  TO  DATE 

1.3.1  Distributed  Mission  Training  (DMT) 

DMT  research  efforts  at  AFRL/HEA  incorporate  both  enhancements  to  VR/VE  technologies  and  development 
of  training  strategies  and  interventions  that  take  advantage  of  these  technologies. 

1.3.2  DMT  Air 

Training  Systems  Technology  Research:  These  efforts  are  focused  around  the  specification  of  Mission 
Essential  Competencies  (MECs)  for  a  selected  group  of  warfighters  such  as  F-16  pilots.  MECs  along  with 
specifications  of  required  knowledge  and  supporting  competencies  describe  the  capabilities  expected  from 
warfighters  that  are  required  to  accomplish  their  missions.  MECs  are  derived  from  detailed  interviews  with 
subject  matter  experts  and  are  used  to  define  the  objectives  for  any  training  event.  Using  MECs  as  the 
foundation,  training  syllabi  have  been  to  develop  individual  and  team  air  combat  skills  for  different  sets  of 
training  objectives  and  levels  of  experience.  DMT  Air  Training  Effectiveness  research  has  expanded  to  focus 
on  coalition  air  operations  through  establishment  of  international  cooperative  research  agreements  with 
Canada  and  the  UK  and  with  Australia.  Coalition,  composite  force  training  missions  have  been  conducted 
with  forces  located  in  Mesa,  AZ,  Toronto  Canada,  and  Bedford,  UK. 

Products  developed  this  year  include: 

•  Procedures  and  analysis  tools  for  crew,  team,  and  individual  cognitive  task  analysis  methods 

•  Mission  Essential  Competency  specification  for  Air-to-Air,  Air-to-Ground,  AW  ACS,  and  SEAD 
Coalition  MECs  for  Air-to-Air,  Air-to-Ground  and  AWACS 

•  Validated  competency-based  syllabi  for  Air  Combat  DMO 

•  Knowledge-  and  skill-based  learning  and  performance  assessment  tools 

•  Automated  grade  sheet  mission  evaluation  metrics 
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•  Scenario-based  performance  assessment  methods 

•  Mission  planning  and  analysis  tools 

•  Cost-benefit  training  utility  trade-off  models 

•  Tools  for  career  field  education  and  training  management  and  reshaping 

1.3.3  High-Resolution  Visuals 

Products  developed  this  year  include: 

•  Assessed  line  rate  and  antialiasing  effects  on  aspect  recognition  using  legacy  testbed  visuals 

•  Integrated  and  evaluated  MetaVR  PC-IG  in  DMT  testbed 

•  Assessed  low  altitude  cueing  in  PC-IG  /  M2DART  for  altitude,  velocity,  and  heading  tasks 

•  Completed  initial  system  resolution  and  antialiasing  research 

•  Assessed  spatial  and  temporal  properties  of  COTS  displays 

•  Developed  digital  interface  for  high-resolution,  laser  projector 

•  Demonstrated  5 120  x  1024  real-time  imagery  on  high-resolution,  laser  projector 

•  Demonstrated  monochrome  holographic  collimating  display 

•  Upgraded  high-resolution,  laser  projector  to  full-color  using  prototype  red  laser 

The  Next  Threat  System:  TNTS  has  been  developed  in  two  versions.  The  ground  version  runs  on  a  PC  and 
provides  EW  simulation  for  distributed  simulation  exercises.  The  airborne  version  is  a  mini  computer,  rack¬ 
mounted  system  that  taps  into  the  aircraft’s  signal  bus  and  provides  EW  simulation  to  on-board  displays. 
The  airborne  system  has  been  installed  and  tested  on  an  MC-130P  aircraft.  Specific  attention  is  paid  to 
accurately  implementing  sensor  scanning  and  target  detection  in  realistic  situations.  Sensor  modeling  is 
designed  to  be  true  to  real  world  performance  in  the  guise  of  beam  pointing.  Beam  pointing  in  the  TNTS  is 
true  to  the  description  in  EWIR.  TNTS  detection  and  tracking  is  allowed  only  within  the  beam  of  the  sensor  as 
the  sensor  scans  (as  applicable)  and  not  based  on  presence  with  a  scan  volume.  The  utilization  of  “real  world” 
clutter  is  limited  by  the  data  base  used  to  describe  the  terrain  in  the  scenario  data  base.  Local  area  clutter  is  to 
be  specified  by  the  scenario  database  using,  initially,  DMA  surface  codes  to  describe  the  clutter  reflectivity 
and  the  correct  geometry  to  correlate  its  effect.  Currently,  such  clutter  is  constrained  to  the  terrain  under  the 
target  or  the  terrain  at  the  point  of  contact  between  the  beam  and  the  surface.  Other  basic  considerations 
include  the  initial  implementation  of  ECM  effects. 

Distributed  Training  Network  Guard:  On  20  February  2003,  the  Air  Force  Research  Laboratory, 
Warfighter  Training  Research  Division  (AFRL/HEA)  in  Mesa  AZ,  successfully  demonstrated  the  capabilities 
of  the  DTNG  via  a  live,  real-time  demonstration.  Attendees  included  more  than  90  DoD  and  DoD  contractor 
personnel  representing  many  of  the  33  mission  training  centers  that  make  up  the  DMO  community. 
In  addition,  system  program  offices  including  ASC,  ESC  and  SMC,  and  Navy  and  Army  representatives 
attended  the  event.  News  of  the  successful  demonstration  piqued  the  interest  of  Mr.  William  Davidson, 
SAF/AA,  who  subsequently  visited  the  site  on  22  May  2003  to  view  a  demonstration  of  the  DTNG  and  to 
discuss  the  future  challenges  of  the  program.  Thus  far,  the  DTNG  has  been  a  success  and  the  Category  I 
Advanced  Technology  Demonstration  is  expected  to  be  closed  by  mid  FY04.  Beyond  the  ATD,  the  DTNG 
will  transition  to  the  MLS  testbed  at  Det  4,  Theater  Aerospace  Command  and  Control  Simulation  Facility, 
Kirtland  AFB  NM,  for  certification  and  accreditation  as  well  as  operational  test  and  evaluation.  Det  4  has  been 
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designated  as  the  DMO  Center  of  Excellence  by  the  CSAF  and  was  chosen  to  lead  the  implementation  of  the 
DMO  concept.  AFRL  will  work  directly  with  Det  4  to  provide  continued  support  for  the  DMO  concept  and 
development  of  the  DTNG.  On  16  July,  the  DTNG  was  briefed  to  the  Top-Secret  and  Below  Interoperability 
Guard  Review  Board  at  the  Defense  Intelligence  Agency  (DIA).  AFRF  was  notified  that  DIA  will  sponsor 
DTNG  for  Certification  and  Accreditation.  Det  4  is  expected  to  achieve  Certification  and  Accreditation  for 
DTNG  by  Mar  of  04. 

Space:  Satellite  Operations  Simulator  (SOpSIM).  The  Satellite  Operations  Simulator  (SOpSim)  concept 
offers  the  first  system-selectable  space  operations  training  system,  scalable  to  a  given  training  environment. 
Utilizing  high  fidelity  visual  representation  and  an  astrodynamically  correct  model  serving  as  the  engine  for 
the  simulation,  SOpSim  brings  to  the  space  operator  a  never  before  available  visually  and  astrodynamically 
correct  space  operations  mission  training  and  rehearsal  tool.  Feveraging  the  DMT  concept  and  technology, 
space  operations  personnel  can  now  train  individually,  as  an  entire  console  crew,  and  up  to  an  entire  Space 
Operations  Center  (SOC).  SOpSim  is  an  innovative  PC-based  simulator  that  addresses  the  mission  training 
and  rehearsal  of  space  operations.  Utilizing  SOpSim,  space  operations  trainers  can  mitigate  risks  historically 
inherent  with  satellite  operations  training.  Having  never  before  had  a  dedicated  training  system,  satellite 
operations  training  was  the  burden  of  on-the-job  training  with  live  mission  systems.  SOpSim  eliminates  the 
risk  associated  with  this  by  taking  the  training  off  line,  and  removing  the  billion-dollar  mission  platform  from 
the  process.  SOpSim  can  now  train  operators  on  traditionally  high-risk  satellite  maneuvers  in  the  low  risk 
environment  of  high  fidelity  simulation.  This  coupled  with  the  first  visualization  of  the  satellite  operations 
significantly  enhances  the  element  of  training  transfer. 

Air  and  Space  Operations  Center:  In  FY03  AFRF/HEA’s  research  team  developed  and  defined  the  first  set 
of  MECs  and  specifications  of  Knowledge,  Skills,  and  Experiences  for  warfighters  in  the  Combat  Operations 
and  Combat  Plans  Divisions  of  the  AOC.  In  FY04,  we  will  use  this  as  a  framework  to  assess  the  current  state 
of  training  in  the  AOC  community.  Through  a  series  of  closely  administered  surveys  we  will  identify  existing 
gaps  in  the  current  training  with  respect  to  the  defined  MECs  required  for  operators  in  those  divisions. 
This  will  provide  a  basis  for  Air  Combat  Command  to  determine  Continuation  Training  requirements  and 
impacts  to  Mission  Qualification  Training  and  Initial  Qualification  Training.  Additionally,  this  will  allow  us 
to  develop  competency-based,  instructionally-principled  training  scenarios  for  simulation-based  training  for 
AOC  warfighters. 

Security  Forces:  Developments  in  the  past  year  include: 

•  Obtained  letter  of  support  for  R&D  from  HQ  USAF/XOF  and  HQ  ACC/SF. 

•  Conducted  training  needs  analysis  with  participation  of  46%  of  active  duty  security  forces  officers; 
direction  of  security  forces  for  air  base  defense  identified  as  high-need  training  area. 

•  Coordination  with  subject  matter  experts  accomplished  at  the  99th  SFS/GCTS  (ACC),  37th  TRG 
(AETC),  and  the  Air  Mobility  Warfare  Training  Center  to  develop  baseline  capability. 

•  Baseline  simulation  capability  developed,  demonstrated,  and  evaluated  for  usability  and  model 
validity  at  96th  SFS/GCTS  and  37th  TRG. 

•  Input  from  field  evaluations  used  for  spiral  development  of  revised  capability. 

•  Revised  simulation  capability  demonstrated  and  evaluated  for  usability  and  model  validity  at  96th 
SFS/GCTS  and  37th  TRG. 

•  Presentations  and/or  technology  demonstrations  delivered  to  HQ  USAF/XOF,  Force  Protection 
Battlelab,  Air  Force  Security  Forces  Center,  and  all  major  command  security  forces  directors. 
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*  Presentations  and  technology  demonstrations  delivered  to  HQ  AFRL/CV,  HQ  AFRL/XPH, 
SAF/AQRT,  OSD/DDR&E,  ARI/IFRU,  and  US  Military  Academy. 

•  Developed  and  submitted  a  proposal  for  evaluation  of  simulation  technology  as  a  means  of  security 
mission  preview;  submitted  proposal  to  AEF  Battlelab,  Force  Protection  Battlelab,  and  Wing 
Commander  at  AL  Udeid  Air  Base. 

•  Independently  developed  and  submitted  an  Education  and  Training  Technology  Application  Proposal 
(ETTAP)  for  field  evaluation  of  SecForDMT  to  the  37th  TRG. 

*  Effort  underway  to  obtain  support  from  HQ  USAF/XOF  for  operational,  test,  and  evaluation  to 
determine  training  value. 

Night  Vision  Training  System  (NVTS):  The  physics  based  approach  used  in  NVTS  has  been  adopted  by  the 
USMC  for  the  Night  Attack  Harrier  (AV-8B)  simulators,  the  USN  F-18  DMT,  the  USAF  F-16  Mission 
Training  Center  and,  National  Training  Center  as  well  as  Weapons  Systems  Trainer  and  Unit  Training  Device 
programs.  The  system  as  developed  for  use  in  Night  Vision  Goggle  (NVG)  training  research  at  AFRL/HEA 
includes  a  correlated  photographic  and  material-classified  database  covers  380  nautical  miles  by  420  nautical 
miles  of  the  Nellis  AFB  training  rang  derived  from  multi-spectral  satellite  imagery,  aerial  photography, 
material  spectral  response  data,  and  Digital  Terrain  Elevation  Data.  The  NVG  sensor  simulation  uses  a 
physics-based  approach  to  provide  an  accurate  in-band,  radiometric  response  for  the  NVG  gain  and  special 
effects  (such  as  halos).  It  senses  at  aperture  radiance  as  defined  by  the  reflectance  and  aspect  of  the  material- 
coded  texel  under  illumination.  As  the  illumination  level  and  angle  change  in  the  simulation,  the  amount  of 
light  reflected  from  each  texel  to  the  viewpoint  changes  in  real  time,  providing  directional  lighting  effects. 
Selected  combat  effects  have  recently  been  modeled  and  include  several  types  of  explosions,  missile  trails, 
flares  and  tracers.  All  of  these  effects  include  near-  and  in-view  effects  for  haloing,  gain,  and  noise.  High- 
resolution  helmet-mounted  displays  present  the  simulation  to  the  user.  These  displays  incorporate  miniature 
cathode  ray  tubes  (CRTs)  mounted  inside  NVG  shells  to  provide  the  same  form,  fit,  and  function  of  actual 
NVGs,  with  the  same  weight  and  center  of  gravity  as  the  NVGs  being  modeled.  The  CRTs  use  the  same 
phosphor  as  current  NVGs  in  order  to  provide  the  same  color  and  decay  characteristics.  Current  Helmet 
Mounted  Displays  have  a  display  resolution  up  to  1700  pixels  by  1350  lines,  non-interlaced,  refreshed  at 
60Hz.  A  Phase  II  Small  Business  Innovative  Research  program  is  developing  a  Helmet  Mounted  Display  to 
support  the  Panoramic  Night  Vision  Goggle.  The  Introductory  NVG  academic  courseware  is  in  use  by  the 
USAF,  USN,  and  USMC.  The  NVG  Instructor’s  course  is  required  for  all  USAF  NVG  instructors. 

Uninhabited  Aerial  Vehicle  Synthetic  Task  Environment:  AFRL/HEA  researchers  have  designed  and 
implemented  a  Synthetic  Task  Environment  (STE)  for  the  Predator  Uninhabited  Air  Vehicle  (UAV). 
This  STE  includes  a  high-fidelity  simulation  of  a  UAV  operator  station  and  provides  a  first  target  domain  for 
HEA’s  computational  process  modeling  of  human  behavior.  With  continuing  basic  research  funding  from 
AFOSR,  PALM  Lab  scientists  are  now  using  this  STE  as  a  research  testbed.  Efforts  to  model  the  behavior  of 
the  UAV  Operator  have  centered  on  basic  maneuvering  -  foundational  skills  required  for  controlling  the 
aircraft.  Later  in  the  research  program,  after  validating  the  basic  maneuvering  model  against  performance 
data,  eye  movement  data,  and  verbal  protocol  data  from  subject  matter  experts,  efforts  will  turn  to  extending 
the  model’s  capabilities.  Eventually,  it  will  represent  the  behavior  of  an  operator  completing  a  simulated 
reconnaissance  mission. 
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1.6  VR  R&D  LABORATORY  FACILITIES 
1.6.1  F-16  DMT  Testbed 

The  F-16  testbed  consists  of  four  high-fidelity,  F-16  Multi-Task  Trainers  (MTTs)  equipped  with  full  field-of- 
view  Mobile,  Modular  Displays  for  Advanced  Research  and  Training  (M2DART)  visual  display  systems, 
two  AWACS  Air  Weapons  Controller  stations,  a  control  and  observation  console,  constructive  threat 
generation  systems,  data  recording  systems,  performance  assessment  systems,  and  two  replay/debrief  systems. 
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Each  M2DART  consists  of  nine,  rear-projection  screens  faceted  around  the  cockpit  providing  the  full  field  of 
view  for  an  F-16  pilot.  The  screens  provide  xx  ft-L  of  brightness  with  yy:l  contrast.  Out-the-window  imagery 
for  two  of  the  cockpits  is  provided  by  SGI  Reality  Monster  Computer  Image  Generators  (CIGs).  Out-the- 
window  imagery  for  other  two  cockpits  is  produced  by  Personal  Computer  (PC)  based  CIGs.  These  CIGs 
manufactured  by  Aechelon  Technologies,  Inc.  and  SDS  International  Inc.  provide  comparable  imagery  to  the 
Reality  Monsters  using  commercial,  off-the-shelf  PC  hardware.  The  control  and  observation  console  consists 
of  ten,  large  display  screens  that  show  the  forward  view  and  selected  cockpit  information  from  all  four  F-16s 
together  with  a  plan-view  of  the  engagement  and  radio  communications.  Cockpit  displays,  radio 
communications,  and  the  plan  view  are  recorded  for  replay  in  one  of  two  debrief  facilities.  Using  the  debrief 
systems,  instructors  can  replay,  fast-forward,  and  rewind  recorded  scenarios  while  zooming  in  and  out  of  the 
plan-view  displays. 

1.6.2  Night  Vision  Training  Laboratory 

The  laboratory  facilities  include  an  NVG  simulation  development  system  comprised  of  an  SGI  Onyx  and  a  PC 
image  generation  system,  a  psychophysics  human  factors  laboratory,  a  precision  equipment  measuring  lab 
including  a  variety  of  NVIS  measuring  and  testing  equipment,  a  multi-media  courseware  development  system 
and  specialized  cameras  for  collecting  NVG  imagery,  an  NVG  academics  training  facility  including 
two  eyelanes,  a  multi-media  classroom,  a  terrain  model  board  and  associated  NVG  instructional  materials. 
The  academic  training  facility  is  located  at  Fuke  AFB. 

1.6.3  High  Resolution  Visual  Displays  Laboratory 

Unique  Facilities: 

•  Dedicated  M2DART  Visual  Testbed 

•  Faser  Projector  Fab 

•  PC-IG,  projector,  and  display  test  lab 

•  PC-IG  systems  from  a  variety  of  vendors 

•  Collimating  and  retro-reflective  displays 

•  Head-mounted  display  facility  (under  development) 

•  DMT  testbed  available  for  follow-on  training  effectiveness  studies 
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Paper  2  -  Army  Research  Institute  (ARI) 

12350  Research  Parkway 
Orlando,  FL  32826 
USA 

Telephone:  (407)  384-3980 
Fax: (407)  384-3999 

E-mail:  Stephen_Goldberg@peostri.army.mil 


Key  Research  Personnel 

Dr.  Stephen  Goldberg,  Dr.  Bruce  Knerr,  Dr.  Michael  Singer,  Dr.  Bob  Witmer, 
Mr.  Donald  Lampton,  Dr.  Larry  Meliza,  Dr.  John  Barnett,  Dr.  Paula  Durlach 


2.1  AREAS  OF  INTEREST 

•  Use  of  VR/VE  for  dismounted  Infantry  training. 

2.2  CURRENT  RESEARCH  PROJECTS 

Virtual  Environment  Research  for  Infantry  Training  and  Simulation.  The  objective  is  to  develop,  integrate, 
demonstrate,  and  evaluate  technologies,  techniques,  and  strategies  for  using  virtual  simulations  for  individual, 
leader  and  small  unit  training,  mission  rehearsal,  concept  development,  and  test  &  evaluation.  Emphasis  is 
being  placed  on  developing  the  capability  within  Virtual  Environments  to  conduct  night,  MOUT  (Military 
Operations  in  Urban  Terrain)  and  contingency  operations.  The  more  recent  experiments  have  also  examined 
instructional  strategies  and  training  interventions  unique  to  training  teams  in  distributed  VEs. 


2.3  RESULTS  ACHIEVED  TO  DATE 

Initiated  a  program  of  experimentation  to  investigate  behavioral  sciences  issues  in  the  use  of  VR  for  military 
training  in  1992.  Following  an  initial  analysis  of  the  task  requirements  for  dismounted  soldier  training,  and  a 
review  of  previous  VR  training  research,  four  experiments  were  conducted  to  investigate  interface  effects  on 
the  capabilities  of  participants  to  perform  simple  tasks  in  VR.  Variables  investigated  included  the  type  of 
control  device,  amount  of  task  practice,  stereoscopic  vs.  monoscopic  helmet-mounted  displays  (FIMDs), 
and  type  of  display  device  (monitor,  Boom,  or  HMD).  Three  experiments  were  performed  that  addressed  the 
effectiveness  of  VR  for  teaching  route  and  configuration  knowledge  of  large  buildings,  and  the  transfer  of  this 
knowledge  to  the  real  world.  The  results  of  these  experiments  led  to  a  program  of  basic  research  on  distance 
estimation  in  VR.  The  next  phase  of  the  research  investigated  the  use  of  VR  to  represent  exterior  terrain  for 
training  both  land  navigation  skills  (identifying  landmarks  and  learning  routes)  and  terrain  knowledge. 
Finally,  research  was  conducted  investigating  the  use  of  VR  for  training  more  complex  tasks.  This  included 
experiments  examining  the  effects  of  self  representation  on  performance,  the  training  of  two-person  hazardous 
materials  teams,  and  distributed  team  training  in  underway.  Overall,  the  program  has  conducted 
16  experiments  involving  over  600  human  subjects.  Knerr  100  et  al.  (1998)  provides  an  overview  of  the 
results  of  the  first  phase  of  the  program  (1993  -  1998),  along  with  recommendations  for  the  use  of  VR  for 
dismounted  soldier  training. 
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Beginning  in  1999,  increasing  emphasis  within  the  program  is  being  placed  on  the  development  of 
technologies  and  techniques  for  the  training  of  Infantry  leader  tasks.  In  2000/2001  we  developed  and  released 
for  free  a  new  library  called  the  Virtual  Environment  Software  Sandbox  (VESS). 


2.4  COLLABORATIVE  PARTNERS 

•  US  Army  Simulation  Training  and  Instrumentation  Command 

•  US  Army  Research  Laboratory,  Human  Research  and  Engineering  Directorate 

•  US  Army  Research  Laboratory,  Computer  and  Information  Sciences  Directorate 

•  US  Naval  Air  Warfare  Center  Training  Systems  Division 

•  University  of  Central  Florida  Institute  For  Simulation  and  Training 


2.5  LITERATURE  PREPARED  BY  RESEARCHERS 
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environment.  (Technical  Report).  Alexandria,  VA:  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences. 

Ehrlich,  J.A.,  Knerr,  B.W.,  Lampton,  D.L.,  and  McDonald,  D.P.  (1997).  Team  situational  awareness  training 
in  virtual  environments:  Potential  capabilities  and  research  issues  (Technical  Report  1069).  Alexandria,  VA: 
U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences. 

Jacobs,  R.S.,  Crooks,  W.H.,  Crooks,  J.R.,  Colburn,  E.,  Fraser,  II,  R.E.,  Gorman,  P.F.,  Madden,  J.L., 
Furness,  III,  T.A.,  and  Tice,  S.E.,  (1994).  Training  dismounted  soldiers  in  virtual  environments:  Task  and 
research  requirements.  (ARI  TR  1011).  Alexandria,  VA:  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences.  (AD  A286  311) 

Knerr,  B.W.  (1996).  Individual  Combatant  Simulation  System  (ICSS)  assessment  plan  (Research  Product 
96-10).  Alexandria,  VA:  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences. 
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Garfield,  K.A.,  Martin,  G.A.,  and  Washburn,  D.A.  (2002).  Virtual  environments  for  dismounted  soldier 
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Witmer,  B.G.  and  Lampton,  D.  (2000).  Simulator  sickness  in  virtual  environments.  Human  Systems 
Information  Analysis  Center  Gateway  Newsletter,  11(2),  pp.  16-17. 

2.6  VR  R&D  LABORATORY  FACILITIES  AVAILABLE 

•  Silicon  Graphics  Workstations  including  an  Onyx  Reality  Engine  Rackmount  System  (3  Graphics  Pipes, 
8  CPUs,  12  RMs,  MCO),  an  Onyx  Reality  Engine  Deskside  (4  CPUs,  4  RMs,  MCO),  an  Onyx 
RealityEngine  Deskside  (2  CPUs,  2  RMs)  a  Crimson  RealityEngine  Deskside  (2  CPUs,  4  RMs,  MCO), 
an  Octane  (2  CPUs.  OCO),  and  several  High  Impact,  02,  Indy,  and  Indigo  Systems. 

•  Evans  and  Sutherland  ESIG  2000  dual  channel  image  generator 

•  SIMNET  8-channel  image  generator  and  M1A1  Tank  Simulator 

•  Numerous  Networked  PCs 

•  Virtual  Research  V8  HMD  (2) 

•  Virtual  Research  VR4  HMD  (3) 

•  Virtual  Research  Flight  Helmet  (2) 

•  VPL  EyePhones 

•  Fake  Space  Labs,  high  resolution  2-color  BOOM 

•  CrystalEyes  (4) 

•  Howlett  CyberFace  II 

•  Ascension  Flock  of  Birds  tracker,  extended  range  with  8  sensors 

•  Ascension  Flock  of  Birds  tracker,  extended  range  with  2  sensors 

•  Ascension  MotionStar  tracker,  extended  range  with  1 6  sensors 

•  Polhemus  trackers  with  multiple  sensors  (3) 

•  LogiTech  Acoustical  trackers  (2) 

•  Crystal  River  Convolvotron 

•  1ST  ChordGloves™  (4  pairs) 

•  1ST  VE  Motion  Treadmill 

•  Mirage  Display  System 

•  Auto  Cad 

•  Alias 

•  ElectroGIG 

•  GMS 

•  GRASS 

•  MultiGen 

•  Neo  Visuals 

•  S-1000 

•  Strata  Vision-3D 
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Paper  3  -  NAVAIR  Orlando  Training  Systems  Division 

12350  Research  Parkway 
Orlando,  FL  32826-3275 
USA 

Telephone:  (407)  380-8168 
E-mail:  Denise.Nicholson@navy.mil 

Key  Research  Personnel 

Dr.  Denise  Nicholson,  Dr.  Robert  Allen,  LT  Richard  Arnold,  Ph.D., 

Stephanie  Lackey  (NAVAIR  Orlando),  LCDR  Dylan  Schmorrow,  Ph.D.,  Pete  Muller  (ONR), 

LT  Joseph  Cohn,  Ph.D.,  Dr.  James  Templeman  (NRL),  Dr.  Rudy  Darken  (NPS) 

3.1  AREAS  OF  INTEREST 

•  VEs  for  the  acquisition  and  use  of  spatial  knowledge 

•  Haptics  (virtual  touch  and  feel),  cross-sensory  display  substitution,  depth  perception 

•  Modeling  of  individual  differences  in  spatial  cognition  at  the  individual  and  team  level 

•  Effectiveness  of  VE  technology  for  small  team  training;  tracking  technology 

•  Locomotion  methods  in  VE 

•  3D  spatial  sound 

3.2  CURRENT  RESEARCH  PROJECTS 

3.2.1  Virtual  Technologies/Environments  (VIRTE)  Demonstration  I;  Expeditionary  Warfare 
Combat  Vehicles  and  Craft 

This  program  is  developing  a  series  of  interoperable,  deployable  combat  simulations  that  support  training, 
mission  planning,  and  rehearsal  in  Expeditionary  Warfare.  By  using  PCs,  and  maximizing  the  use  of 
Government-owned  software,  the  simulations  will  be  deployable,  inexpensive,  and  easy  to  maintain. 

The  three  Expeditionary  Warfare  Platforms  that  were  developed  are  the  Landing  Craft,  Air  Cushion  (LCAC), 
the  Expeditionary  Fighting  Vehicle  (EFV)  (previously  known  as  the  Advanced  Amphibious  Assault  Vehicle 
(AAAV)),  and  the  helicopter.  Each  of  these  platforms  offers  unique  simulation  and  training  challenges. 
The  goal  of  the  program  is  to  demonstrate  Human-centric  designs  that  meet  a  variety  of  challenging  goals. 
VIRTE  simulations  will  be  able  to  form  a  federation  using  DoD’s  High-Level  Architecture  (HLA)  and  interact 
in  a  highly  complex  synthetic  environment  consisting  of  realistic  weather  (rain,  wind,  fog,  etc.),  ocean 
environment  (wave  height,  sea  state,  surf  zone,  etc.),  man-made  features  (buildings,  obstacles,  craters,  etc.), 
and  other  vehicles. 

3.2.2  Virtual  Technologies/Environments  (VIRTE)  Demonstration  II;  Close  Quarters  Battle 
for  Military  Operations  in  Urban  Terrain  (CQB  for  MOUT) 

VIRTE  Demonstration  2  is  developing  the  technologies  and  infrastructure  to  allow  Marines  and  Seals  to 
interact  with  the  virtual  environment  in  a  more  realistic  and  natural  manner  than  is  currently  possible. 
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By  tracking  the  entire  body,  instead  of  just  the  head  and  the  weapon,  the  avatar  in  the  virtual  world  will  be 
much  more  realistic.  Advances  in  Head  Mounted  Displays  (HMDs)  will  allow  greater  immersion  and 
improved  realism.  The  simulation  infrastructure  will  allow  much  more  natural  interaction  with  the  virtual 
world.  For  example,  a  Marine  or  Seal  will  be  able  to  kick  open  doors,  move  furniture  with  his  body,  and 
observe  realistic,  physics  based  weapons  effects. 

Today’s  infantry  must  quickly  adapt  to  diverse  situations,  from  high-intensity  warfare  to  peacekeeping 
missions,  functioning  in  small  teams  and  with  increasing  physical,  emotional,  and  intellectual  demands. 
Due  to  reduced  live  training  opportunities,  VIRTE  will  need  to  provide  a  level  of  visual,  audio,  and  haptic 
fidelity  that  has  not  been  available  in  a  single  synthetic  exercise.  Warrior  acceptance  of  such  advanced 
training  technology  will  require  detailed  effectiveness  data  developed,  collected  and  analyzed  by  a  thorough 
human  centered  design  process,  with  continuous  user  input  into  spiral  developments,  then  tested  in  training 
transfer  studies. 


3.3  RESULTS  ACHIEVED  TO  DATE 

3.3.1  VIRTE  Demonstration  I,  Combat  Vehicles  and  Craft 

VIRTE  Demonstration  I  developed  three  unique  interoperable  Expeditionary  Warfare  simulations  in  FY  02 
and  FY  03.  They  share  a  common  networked  synthetic  battlespace,  but  they  are  very  different  in  their 
technologies  and  applications.  The  three  Expeditionary  Warfare  platforms  that  were  chosen  are  the  Landing 
Craft,  Air  Cushion  (LCAC),  the  Expeditionary  Fighting  Vehicle  (EFV)  (previously  known  as  the  Advanced 
Amphibious  Assault  Vehicle  (AAAV)),  and  the  helicopter.  The  VELCAC  uses  a  gaming  engine  (Gamebryo) 
to  develop  high  fidelity  virtual  cockpit  for  all  three  crew  positions.  The  architecture  allows  a  single  crew  to 
train  in  their  position  and  supports  team  training  of  the  entire  crew.  The  VEEFV  uses  a  medium  fidelity 
mockup  of  the  interior  of  the  vehicle  and  provides  gunnery  and  team  training.  The  same  software  can  be  used 
for  Embedded  Training  and  also  supports  laptop  operations.  The  VEHELO  uses  a  Chromakey  approach  and  a 
HMD  to  combine  a  low  fidelity  physical  mock  up  with  the  virtual  world.  It  also  supports  a  laptop 
configuration. 

3.3.2  VIRTE  Demonstration  II,  Close  Quarters  Battle  for  Military  Operations  in  Urban 
Terrain  (CQB  for  MOUT) 

A  prototype  testbed  has  been  developed  at  NRL  and  it  has  been  used  to  test  and  integrate  component 
technologies.  A  government-owned,  PC  based,  HLA  compliant,  game  quality  simulation  infrastructure  has 
been  developed  and  demonstrated.  A  government-owned  3D  spatial  sound  software  package  has  been 
developed  and  integrated  into  the  simulation  architecture.  Real-time  control  by  an  individual,  using  only  their 
body,  of  a  virtual  avatar  has  been  demonstrated.  Various  methods  of  locomotion  in  VE  have  been  tested. 
Magnetic,  passive  optical,  and  active  optical  tracking  systems  have  been  tested  and  integrated.  The  new 
PhaseSpace  tracking  system  is  the  result  of  an  ONR  funded  SBIR. 


3.4  COLLABORATIVE  PARTNERS 

•  ONR  (Office  of  Naval  Research) 

•  NPS  (Naval  Post  Graduate  School) 

•  NRL  (Naval  Research  Laboratory) 
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•  ARI  (Army  Research  Institute) 

•  U.S.  Army  Program  Executive  Office  for  Simulation,  Training  and  Instrumentation  (PEO  STRI) 

•  U.S.  Army  Research  Development  and  Engineering  Command  (RDECOM) 
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3.6  VR  R&D  LABORATORY  FACILITIES  AVAILABLE 

NAVAIR  Orl 

Prototypes  of  VELCAC,  VEEFV,  VEHelo,  and  ISMT-E  are  available  for  demonstration. 

NRL 

VIRTE  Demo  2  prototype  using  VICON  tracking  system;  PhaseSpace  Tracking  System. 

NPS 

See  section  7F  for  details. 


RTO-TR-HFM-1 21  -Part-1 


7-33 


HUMAN  FACTORS  ISSUES  IN  THE  USE  OF  VIRTUAL 

AND  AUGMENTED  REALITY  FOR  MILITARY  PURPOSES  -  USA 


7-34 


RTO-TR-HFM-1 21  -Part-1 


HUMAN  FACTORS  ISSUES  IN  THE  USE  OF  VIRTUAL 
AND  AUGMENTED  REALITY  FOR  MILITARY  PURPOSES  -  USA 


Paper  4  -  Naval  Postgraduate  School 

The  MOVES  Institute 

Graphics  and  Video  Laboratory,  Dept,  of  Computer  Science 
Code  CS/ZK,  Monterey,  CA  93943 
USA 

Telephone:  (408)  656-2305 
Fax:  (408)  656-2814 

E-mail:  zyda@movesinstitute.org 

Key  Research  Personnel 

Michael  Zyda,  Don  Brutzman,  Rudy  Darken,  John  Hiles,  Ted  Lewis,  Alex  Mayberry, 
Russell  Shilling,  Joe  Sullivan,  Alex  Callahan,  and  Margaret  Davis 


4.1  AREAS  OF  INTEREST 

The  MOVES  Institute’s  mission  is  research,  application,  and  education  in  the  grand  challenges  of  modeling, 
virtual  environments,  and  simulation.  Specialties  are  3D  visual  simulation,  networked  virtual  environments, 
computer-generated  autonomy,  human-performance  engineering,  immersive  technologies,  defense/entertainment 
collaboration,  and  evolving  operational  modeling. 


4.2  CURRENT  RESEARCH  PROJECTS 

4.2.1  3D  Visual  Environments  -  Technical  Director:  Dr.  Don  Brutzman 

NPS  has  been  instrumental  in  standardizing  the  extensible  3D  (X3D)  graphics  specification,  collaborating 
with  the  Web3D  Consortium,  ISO,  and  WWW  Consortium  to  produce  a  synthesis  of  the  Virtual  Reality 
Modeling  Language  (VRML)  and  the  Extensible  Markup  Language  (XML)  to  exploit  X3D  graphics  for  Web- 
based  viewing  of  3D  scenes.  The  addition  of  geospatial  representations,  humanoid  animation,  DIS  networking 
capabilities,  advanced  3D  rendering,  computer-aided  design  interchange,  and  other  capabilities  makes  3D 
graphics  and  visualization  broadly  available  on  the  Web. 

In  scenario  authoring  and  visualization  for  advanced  graphics  environments  (SAVAGE),  we  have  built 
sophisticated  open-license  military  models  in  X3D  as  part  of  the  SAVAGE  project.  Dozens  of  students  have 
contributed  high-res  models  for  ships,  aircraft,  submarines,  land  vehicles,  robots,  humanoid  behaviors, 
environmental  effects,  etc.  The  SAVAGE  archive  is  a  multiple-CD  set  documenting  over  700  military  models, 
scenarios,  theses,  etc. 

Autonomous  underwater  vehicle  visualization  is  also  a  MOVES  concern.  NPS  provides  sophisticated 
capabilities  in  modeling  and  visualizing  oceanographic  data  collected  by  underwater  robots.  With  Office  of 
Naval  Research  (ONR)  sponsorship,  we  have  participated  in  fleet  battle  experiments  to  assess  minefield- 
clearance  by  robots,  and  translate  telemetry  and  communicate  messages  to  the  global  Command-and  Control 
System  (Maritime)  mine -warfare  environmental-database-analysis  library. 
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In  XML  for  operations  orders,  we  can  auto-generate  large-scale  VEs  corresponding  to  regional 
(and  potentially  theatrical)  operations.  This  new  capability  can  be  communicated  via  existing  message 
circuits,  but  for  actual  deployment,  a  common  vocabulary  with  common  semantics  is  needed;  NATO’s 
battlespace  generic  hub  (BGH)  appears  a  good  candidate  for  such  a  model.  We  are  showing  how  BGH  can  be 
expressed  in  XML  for  modeling  joint  and  coalition  tactical  scenarios.  This  work  is  seen  as  necessary  for 
enabling  worldwide  battlespace  presence,  monitoring,  and  visualization.  Applications  to  homeland  defense 
and  assessment  of  the  effects  of  weapons  of  mass  destruction  provide  further  challenges. 

DoD  modeling  and  simulation  (M&S)  must  identify  and  adopt  transformational  technologies  of  direct  tactical 
relevance  to  warfighters.  The  only  software  systems  that  composably  scale  to  worldwide  scope  utilize  Web 
technologies;  therefore  an  extensible  Web-based  framework  offers  promise  in  scaling  up  M&S  systems  to 
benefit  training,  analysis,  acquisition,  and  operational  warfighters.  We  are  defining  an  extensible  modeling 
and  simulation  framework  (XMSF)  to  exploit  Web-based  technologies.  Government,  academic,  and  industrial 
experts  are  working  under  investigators  from  the  Naval  Postgraduate  School,  George  Mason  University,  SAIC 
and  Old  Dominion  University.  XMSF  is  a  composable  set  of  standards,  profiles  and  recommended  practices 
for  Web-based  modeling  and  simulation.  XML-based  markup  languages,  Internet  technologies,  and  Web 
services  will  enable  a  new  generation  of  distributed  M&S  applications  to  emerge,  develop  and  interoperate. 
The  precepts  of  XMSF  are: 

•  Web-based  technologies  applied  within  an  extensible  framework  will  enable  a  new  generation  of 
M&S  applications  to  emerge,  develop  and  interoperate. 

•  Support  for  operational  tactical  systems  is  a  missing  requirement  for  such  M&S  applications 
frameworks. 

•  An  extensible  framework  of  XML-based  languages  can  bridge  forthcoming  M&S  requirements  and 
open/commercial  Web  standards,  while  supporting  existing  M&S  technologies. 

•  Compatible,  complementary  technical  approaches  are  now  possible  for  model  definition,  simulation 
execution,  network-based  education,  network  scalability,  and  2D/3D  graphics  views. 

•  Web  approaches  for  technology,  software  tools,  content  production  and  broad  use  provide  best 
business  cases  from  an  enteiprise-wide  (worldwide)  perspective. 

4.2.2  Networked  Virtual  Environments  -  Technical  Director:  Dr.  Don  Brutzman 

MOVES  continues  research  in  networked  virtual  environments  through  NPSNET-V,  a  platform  for 
investigating  new  concepts  in  related  design.  It  features  composable  components,  and  can  be  extended  at 
runtime;  it  is  graphics-standard  agnostic,  and  can  use  entirely  new  network  protocols  loaded  at  runtime.  It  has 
been  used  to  test  new  ideas  in  interest  management,  security,  and  dynamic  extensibility. 

Cross-format  schema  protocol  (XSFP)  is  a  technique  for  saving  XML  data  in  binary  format.  XML  data  can  be 
read  by  many  platforms,  but  it  is  verbose,  storing  all  data  as  strings.  XSFP  uses  XML  in  bandwidth-  or 
storage-constrained  environments,  increasing  processing  speed  in  data-intensive  XML  documents.  MOVES  is 
developing  DIS  libraries  in  many  contexts  (e.g.  Java,  X3D,  and  browser  plugins),  and  describing  DIS  protocol 
in  terms  of  XML  and  XSFP. 

4.2.3  Computer  Generated  Autonomy  -  Technical  Director:  John  Hiles 

In  2002,  the  MOVES  Institute  established  the  Center  for  the  Study  of  Potential  Outcomes  to  employ  our 
connector-based,  multi-agent  systems  (CMAS)  concept  and  cognitive  science  in  modeling  terrorist  behaviors. 
The  first  project  of  the  center  is  Project  IAGO. 
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Project  IAGO  (integrated,  asymmetric,  goal  organization)  aims  to  develop  a  conceptual  model  and  prototype 
implementation  of  a  cognitive  model  of  the  decision-making  and  dynamic  behaviors  of  terrorists.  The  object 
is  to  develop  an  anti-terrorist  tool  for  intelligence  analysts,  enabling  them  to  explore  a  complex,  hyper¬ 
dimensional  space  of  terrorist  capabilities  and  possibilities  in  a  social  space,  with  potential  for  identifying 
infrastructure  vulnerabilities  and  detection/prevention  opportunities.  This  work  also  explores  benefits  and 
limitations  in  applying  the  CMAS  concept,  n  approach  conceived  by  MOVES  professor  John  Hiles  and 
inspired  by  the  information  exchange  and  -processing  techniques  that  have  evolved  at  the  cellular  level. 

Initially,  IAGO  will  construct  a  prototype  cognitive  model  for  exploring  terrorist  behaviors,  in  three  phases: 

•  Phase  1:  Demonstration  Model.  Initial  design  and  implementation  to  generate  software  components 
that  capture  fundamental  CMAS  concepts  of  tickets,  connectors,  and  templates.  Lessons  from  phase 
one  provide  the  software  foundation  for  phase-two  activities. 

•  Phase  2:  Proof-of-Concept  Model.  This  phase  produced  an  initial  prototype  implementation  of 
mental  blending,  a  cognitive-psychological  model  of  creative  thought  in  which  perceptions  and 
concepts  are  combined  under  the  guidance  of  generic  problem-solving  mental  spaces  and  goal 
motivations  to  create  cognitive  blends  (see  cognitive  training  agents  below).  Development  is  ongoing 
to  yield  more  complex  blended  spaces  from  the  initial  mechanisms. 

•  Phase  3:  Domain  Demonstration.  The  next  phase  is  application  and  demonstration  of  the  blended- 
mental-space  modeling  approach  to  represent  influences  and  dynamics  in  forming  terrorist  behaviors 
leading  to  decisions  and  actions  and  to  show  the  potential  of  the  approach  as  a  useful  model  for 
analysts.  Early  work  with  experts  has  resulted  in  a  characterization  of  decision  elements  influencing 
terrorist  behaviors.  This  information  enables  formulation  of  initial  generic  spaces  and  goals  for 
interplay  with  an  information  stream,  to  produce  behaviors  comparable  to  real-world  actions. 

While  our  funding  for  IAGO  was  small  in  FY2002,  the  project  shows  great  promise.  In  2002,  the  Defense 
Threat  Reduction  Agency  contracted  with  the  MIIS  Center  for  Non-proliferation  Studies  for  a  literature 
review  of  existing  terrorist-behavior  models.  Two-hundred-and-sixty-five  models  were  studied,  and  IAGO 
emerged  in  the  top  ten. 

Cognitive  training  agents  are  also  under  intensive  research.  In  January  2003,  the  MOVES  computer-generated 
autonomy  team  reached  a  milestone.  Using  a  multi-agent  system  combined  with  CMAS  technology  we 
developed  over  the  last  three  years,  we  demonstrated  that  our  software  could  do  what  cognitive  psychologists 
call  cognitive  blending,  producing  cognitive-integration  networks.  That  is,  our  software  was  able  to  create 
new  knowledge  in  situ  based  on  what  it  was  doing  and  what  it  wanted  to  accomplish.  Until  this  time, 
advanced  multi-agent  work  has  only  been  able  to  demonstrate  the  ability  to  adaptively  explore  problems  with 
intent  and  purpose.  Software  blending  means  that  multi-agent  software  can  now  be  built  to  answer  questions 
such  as,  “What  do  you  know?”  “How  do  you  know  that?”  or  (most  importantly  for  training),  “What  are  you 
doing?” 

In  the  next  year,  we  intend  to  extend  these  experimental  results  to  create  adjustable  cognitive  training  agents 
that  will  add  new  capabilities  to  training.  The  cognition  of  these  agents  will  be  built-in  and  applicable  to  a 
range  of  applications.  Specialization  will  take  place  outside  these  capabilities  (for  example,  a  specific 
application  will  require  specific  input  and  output  routines  (sometimes  referred  to  as  input  and  actuator  suites), 
and  an  application-specific  set  of  meta-data  packages  that  we  call  generic  spaces,  which  would  describe  the 
types  of  cognitive  operations  needed  in  that  particular  area). 
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Training  software  equipped  with  cognitive-training  agents  could  ultimately  take  on  the  following  capabilities: 

•  Training  involving  competition  with  virtual  opponents  would  add  adversaries  that  adapt,  deceive,  and 
could  explain  what  they  were  doing  and  why. 

•  Training  for  tasks  not  involving  problem  solving  could  self-adjust  to  press  students  to  their  limits  of 
understanding,  then  offer  explanations  of  their  boundaries  and  what  additional  work  might  expand 
them.  In  this  and  the  following  training  applications,  cognitive  training  agents  guide  the  training 
software  rather  than  act  as  adversary. 

•  Training  for  problem  solving  where  innovation  and  discovery  are  demanded  could  continually  alter 
the  situation  to  keep  it  open  and  freshly  challenging. 

Not  all  training  software  would  benefit  from  these  properties.  But  important  benefits  can  be  derived  from 
software  that  adapts  and  incoiporates  discoveries  into  knowledge  that  it  produces  as  it  goes. 

4.2.4  Human-Performance  Engineering  -  Technical  Director:  Dr.  Rudy  Darken 

We  find  two  fundamental  barriers  to  achieving  the  transformation  in  training  the  CNO  has  called  for: 
(1)  the  cost  of  simulation  for  training  is  too  high,  and  (2)  the  development  cycle  from  mission-need  to  product 
is  too  long.  Low-cost  simulators  and  the  bringing  training  to  the  fleet  swiftly  will  have  an  immediate  impact 
on  the  individual  warfighter.  HPE  has  been  focusing  on  these  issues  through  the  VIRTE  program  to  develop 
high-fidelity  deployable  trainers. 

Our  work  includes  the  development  of  a  Chromakey-  augmented  training  environment,  whereby  the  near-field 
cockpit  of  a  helicopter  is  captured  with  a  camera  and  mixed  with  a  simulated  “out  the  window”  view,  allowing 
inexpensive  reconfigurable  training  to  occur.  Similarly,  we  are  developing  a  “suitcase”  simulator  for  close- 
quarters  battle  for  the  second  phase  of  VIRTE,  miniaturizing  as  many  components  as  possible  to  facilitate 
shipboard  usage.  This  will  be  integrated  with  our  forward-observer  trainer  towards  a  full  spectrum  combined- 
arms  training  capability.  A  common  theme  is  the  use  of  open  standards  and  computer  gaming  technologies  to 
increase  performance  while  driving  down  development  costs.  All  simulations  for  training  developed  in  our 
laboratory  undergo  thorough  testing  both  in  house  and  in  the  fleet  to  determine  their  value  in  transfer  of 
training. 

As  a  part  of  DARPA’s  augmented-cognition  (Aug-Cog)  program,  MOVES  is  researching  a  context  machine 
to  “improve  the  performance  of  the  human-machine  symbiosis  by  an  order  of  magnitude  or  greater,”  thereby 
contributing  to  Aug-Cog’s  goal  of  improving  warrior/computer  interactions,  advancing  systems-design 
methodologies,  and  re-engineering  military  decision-making  at  a  fundamental  level.  MOVES  is  exploring  the 
computer  science  behind  creating  a  system  able  to  determine  a  situation’s  context  and  thereby  assist  in 
accomplishing  its  goals.  Using  a  game  engine  to  simulate  real-world  inputs  and  provide  a  means  of  output, 
MOVES  has  built  a  prototype  system  and  continues  research  into  the  artificial  intelligence  and  other  factors 
required  to  determine  context  and  act  upon  it. 

In  addition,  MOVES  has  evaluated  the  training  effectiveness  of  a  damage-control  trainer  built  at  University  of 
Illinois,  Urbana-Champaign.  We  evaluated  the  system  on  surface-warfare  officers  stationed  at  NPS  to 
understand  the  efficacy  of  the  system  in  training  DCAs.  We  expect  to  present  our  findings  at  ONR  Workshop 
in  May.  Funding  for  HPE  has  been  provided  by  ONR,  N61M,  DARPA,  and  the  FAA. 
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4.2.5  Immersive  Technologies  -  Technical  Director:  LCDR  Russ  Shilling,  USN 

The  immersive  technologies  directorate  met  major  milestones  in  2002.  The  goal  has  been  not  only  to  apply 
virtual  environment  and  video-game  technology  to  training  tasks,  but  also  bridge  the  gap  between  these 
technologies  and  the  operational-  warfare  environment. 

First,  technologies  were  advanced  for  the  development  of  a  sourceless  postural  tracking  system  using 
magnetic  and  inertial  sensors  to  accomplish  full-body  tracking  in  a  virtual  environment.  The  unique  and 
exciting  part  of  this  technological  advance  centers  on  a  tracking  device  only  slightly  larger  than  a  quarter. 
Patents  for  this  device  are  measured  the  physiological  impact  of  VE  technology  and  emotion/arousal  on 
cognitive  abilities  in  a  training  task. 

Finally,  using  the  student-built  cave  system  and  other  visual  technologies,  we  are  combining  entertainment 
techniques,  video-game  technology,  and  advanced  display  design  to  solve  problems  associated  with 
information  management  in  network-centric  warfare  tasks,  especially  in  command  and  control  (video  games 
routinely  use  various  strategies  to  allow  players  to  track  and  manipulate  hyper-dimensional  data  within  game 
play). 

We  will  see  if  some  of  these  same  strategies  can  be  applied  to  helping  planners,  analysts,  and  operators  track 
multi-dimensional  data  sets  associated  with  the  fusion  of  large  amounts  of  tactical  data  from  different  sources 
in  a  live  warfare  setting.  At  the  same  time,  we  will  be  examining  game  engines  and  editors  to  see  if  traditional 
methods  of  war  gaming  might  be  better  implemented  or  improved  using  game-engine  technology.  Results  of 
this  research  will  be  presented  at  ForceNet  in  April  2003. 

4.2.6  Evolving  Operational  Modeling  -  Technical  Director:  LCDR  Alex  Callahan,  USN  (ret) 

The  technical  directorate  for  evolving  operational  modeling  became  established  as  the  configuration  manager 
for  the  naval  simulation  system  (NSS),  an  analytical  model  with  unique  capabilities  for  representing  network¬ 
centric  warfare. 

The  directorate  coordinated  efforts  between  SPAWAR  Systems  Center,  San  Diego,  developers  and  testing 
agencies  to  ensure  the  quality  of  delivered  versions,  and  continued  to  nurture  a  broad  base  for  operational 
analysis  across  government,  military,  and  commercial  interests,  with  NSS  as  the  focal  point. 

Evolving  Operational  Modeling  obtained  tasking  in  several  key  areas  of  NSS  employment,  including  analysis 
of  alternative  platforms  for  the  multi-mission  aircraft  program.  The  directorate  prepared  draft  curriculum 
materials  for  a  new  course,  applied  combat  modeling,  providing  insights  into  the  application  of  combat 
models  (using  NSS  as  the  exemplar)  to  military  operational  analysis. 

The  directorate  led  a  working  group  of  faculty  from  operations  research  and  MOVES  to  review  the  combat 
modeling  curriculum  at  NPS.  As  a  result  of  these  efforts,  existing  combat  modeling  courses  have  been  revised, 
greater  infrastructure  has  been  provided  for  the  war-gaming  analysis  course,  and  a  school-wide  wargaming 
policy  and  advisory  committee  has  been  established  to  revitalize  application  of  warfare  gaming  across 
multiple  disciplines. 

4.2.7  Defense/Entertainment  Collaboration  Creative  Director:  Alex  Mayberry 

The  MOVES  Institute  has  been  in  the  press  continually  with  our  America ’s  Army  project  (see  Appearances, 
below).  Newsweek  has  toasted  “the  legendary  Naval  Postgraduate  School”  in  the  aftermath  of  AA  and  its 
success,  and  we  will  soon  be  in  Newsweek  again. 
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AA  has  engendered  much  faculty  and  master’s-student  interaction,  yielding  several  completed  theses  this  year, 
and  has  brought  NPS  much  positive  media  attention.  It  has  inspired  the  CNO  to  task  the  Naval  War  College’s 
strategic  studies  group  (SSG)  to  perform  a  study  on  the  utility  of  massively  multi-player  gaming  as  the  basis 
for  the  development  of  future  large-scale  M&S  systems.  That  study  reports  out  to  the  CNO  in  July  2003. 
The  MOVES  director  gave  a  presentation  on  the  project  to  the  SSG  in  December  2002  for  that  study.  The  last 
two  SSG  plenary  meetings  have  discussed  massively  multi-player  gaming. 

AA  is  highly  approved  by  the  sponsor  for  its  transformation  of  Army  recruiting.  As  of  the  3rd  of  September 
2003,  there  were  2M+  registered  players  of  AA,  with  1.3M+  having  completed  basic  combat  training  in  the 
game.  Over  2 1 8M+  game  missions  have  been  completed,  and  some  1 00K  gaming  hours  per  day  are  played. 

To  understand  the  dividends  of  the  game  from  the  US  Army  perspective,  a  look  at  traditional  recruiting  is 
in  order.  The  army  spends  $2B  (two  billion)  per  year  to  attract  and  enlist  120,000  recruits  (80,000  army, 
40,000  national  guard).  That’s  $16,666  per  soldier. 

Twenty  percent  (or  24,000)  of  these  recruits  drop  out  during  basic  combat  training  with  the  excuse  that  the 
army  was  not  what  they  expected  and  combat  training  was  not  for  them.  With  them  goes  $400M  in  wasted 
recruiting  expenditure.  In  addition,  the  army  has  spent  $75K  each  for  training;  thus,  the  army’s  loss  per  annum 
from  this  dropout  group  is  $2.2  billion. 

America ’s  Army  cost  $7M  to  build  over  the  first  twenty-four  months,  a  tag  equivalent  to  that  of  420  recruits 
who  wash  out  (if  we  count  recruiting  costs  alone).  If  the  game  encourages  only  120  potential  waverers  to  stick 
with  it,  it’s  broken  even,  counting  recruiting  and  training  costs.  And  of  course,  if  it  attracts  those  who  would 
not  otherwise  have  considered  an  army  career,  it’s  worth  $92K  apiece. 

The  Army  estimates  AA  has  the  potential  to  save  some  $700M-$4B  per  year.  With  respect  to  recruitment, 
actual  results  won’t  be  known  for  four  or  five  years,  when  the  current  raft  of  thirteen-  and  fourteen-year  olds 
will  be  old  enough  to  join.  The  hope  is  that  through  realistic  role  playing  and  exploration  of  a  soldier’s  job,  the 
important  work  of  the  military  will  be  among  the  options  that  compatible  young  men  and  women  will  consider 
when  planning  a  career. 

Improvements  are  continually  made.  By  August  2003,  occupations  within  the  game  will  include  infantry, 
medic,  engineers,  RSTA/Scouts,  and  Special  Forces: 

•  Medic/9 1W  &  combat  lifesaver:  Four  missions,  from  AIT  at  Brooke  Army  Medical  Center  through 
an  STX  under  field  conditions.  These  missions  are  pass-fail  and  enact  expert  information  on  combat 
lifesaving.  This  training  conveys  lifesaving  information  applicable  to  the  population  for  homeland 
defense.  One  mission  will  incorporate  training  to  recognize  the  symptoms  of  nerve  agent  as  well  as 
immediate  self-  and  buddy-aid  for  nerve-agent  casualties  (funded  by  FORSCOM). 

•  Special  Forces:  Several  missions  to  replicate  the  Robin  Sage  exercise  as  apart  of  SFAS  with  emphasis 
on  land  navigation  and  escape  and  evasion.  These  missions  will  qualify  players  to  enter  specialized 
S.F.  training  and  be  assigned  in  multiplayer  S.F.  missions. 

New  units  and  weapons  added  into  AA: 
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Stryker:  the  Stryker  debuted  in  May  in  a  transport-  and  support-by-fire  role  within  a  new  online, 
multiplayer  mission.  Coverage  of  the  SBCT  within  the  Game  expanded  throughout  the  summer  of 
2003. 
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*  TACOM-ARDEC  is  funding  incorporation  of  the  objective  individual  combat  weapon  (OICW)  and 
the  shoulder-launched  multipurpose  assault  weapon/bunker-defeat  munition  (SMAW-D)  into  the 
game. 

Having  a  successful  online  game  inside  the  MOVES  Institute  is  like  having  your  own  particle  accelerator. 
Lots  of  proposed  applications  and  interesting  research  are  coming  in  the  door. 

Many  related  training  applications  using  the  AA  code  base  as  a  starting  point  are  being  considered.  We  have 
funding  from  one  project  that’s  using  Operations  for  treaty  verification  pre-planning,  and  an  Air  Force  group 
is  looking  at  funding  a  training  level  within  the  game  that  will  deal  with  force  protection. 

Infantry  soldiers  at  Fort  Benning  are  using  Operations  before  setting  foot  on  the  real  range.  Also,  the  Army’s 
objective  force  is  having  us  integrate  prototypes  of  their  new  weapons  systems  into  Operations  to  evaluate 
their  potential  utility.  We  are  building  special  levels  of  the  game  for  the  Special  Forces,  both  for  recruiting  and 
SF  training. 

We  have  strong  interest  from  Commander  Naval  Surface  Forces  Pacific  in  our  building  a  game  for  material- 
assessment  training.  They  have  approved  both  proposal  and  schedule  and  are  raising  funding  for  the  project. 
One  extraordinary  possibility,  raised  by  the  undersecretary  of  defense’s  office,  is  massively  multiplayer 
(MMP)  gaming.  The  AA  project  is  being  looked  at  both  as  a  model  of  how  such  an  effort  could  be  carried  out 
within  government  and  as  possible  starting  point  for  a  MMP  project.  The  work  involved  might  include  the 
procurement  (or  development)  of  a  government-owned  game  engine  capable  of  full-spectrum  combat 
modeling  and  large-scale  inter-operability  integration,  as  well  as  a  programming  interface  for  modeling 
individual  and  organizational  behaviors  and  stories. 

An  additional  goal  would  be  a  rapid  prototyping  interface  to  the  MMP  that  would  allow  any  mission  to  be  put 
together  nearly  overnight. 

4.3  RESULTS  ACHIEVED  TO  DATE 

•  In  3D  visual  simulation  and  networked  virtual  environments,  we  have  created  the  extensible  modeling  and 
simulation  framework  (XMSF),  an  effort  cited  as  the  most  important  strategy  for  connecting  DoD 
modeling  and  simulation  to  C4I  systems. 

•  Our  terrorist  behavior-modeling  effort,  Project  IAGO,  is  listed  among  the  top  ten  in  a  DTRA  survey  of 
two  hundred  and  sixty  five  models. 

•  Our  Chromakey  augmented  training  environment  has  been  deployed  to  helicopter  squadron  10  (HS-10) 
to  study  its  utility  in  flight  navigation  training. 

•  Our  achievements  in  immersive  technology  include  a  pending  patent  for  our  inertial  tracker.  Sounds  we 
and  LucasFilm’s  Skywalker  Sound  recorded,  of  an  LCAC  for  a  Marine  Corps  training  VE,  were  used  for 
a  hovercraft  in  the  movie  Minority  Report. 

•  Our  America ’s  Army  is  the  fastest-growing  online  game  ever,  and  has  won  or  been  runner-up  for  several 
best-game-of-the-year  awards.  The  project  is  cited  as  a  transformational  model  for  turning  the  PC  game 
into  a  communications  medium  and  demonstrating  how  innovative  projects  can  succeed  within  DoD. 
AA  is  expected  to  save  $700M  to  $4B  annually  and  has  inspired  the  CNO’s  strategic  studies  group  to 
consider  massively  multi-player  gaming  for  combat  modeling;  other  defense  agencies  are  pursuing  similar 
studies.  The  game  is  the  first  successful  defense/entertainment  collaboration,  as  spelled  out  in  the  National 
Research  Council  report,  “Modeling  and  Simulation  -  Linking  Entertainment  and  Defense.” 
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•  We  have  applied  techniques  from  VE  and  entertainment  to  enhance  comprehension  of  complex  tactical 
information  in  “live”  command-and-control  settings.  We  showed  that  radio,  radar,  air-traffic  control,  and 
possibly  UAV  communications  could  be  improved  using  spatialized  cues  over  headphones,  presenting 
results  at  the  ForceNet  2003  conference. 

We  support  our  students  through  courses  and  funded  research  directly  related  to  our  mission.  Our  projects 
provide  DoD-  and  DoN-relevant  thesis  topics  for  officer  students.  Funded  projects  indicate  serious  interest  in 
our  research  and  educational  abilities.  In  FY2002,  MOVES  had  $1 1.4M  in  reimbursable  funding.  As  of  mid- 
FY2003,  we  had  some  $12M  in  reimbursables  (forty-four  accounts  from  thirteen  sponsors). 

MOVES  has  expanded  greatly,  currently  employing  sixty-eight  faculty  and  staff.  Students  working  in  institute 
projects  increased  from  forty  to  sixty-eight,  hailing  from  twelve  curricula  (MOVES,  CS,  OR,  IT,  IS,  NSA, 
IW,  meteorology,  ME,  ECE,  UW  and  C4I);  see  theses  on  our  website. 

4.4  COLLABORATIVE  PARTNERS 

4.4.1  Civilian 

4.4.1. 1  Academic 

•  Boston  College 

•  California  Polytechnic  State  University 

•  Camegie-Mellon  University 

•  California  State  University,  Monterey  Bay 

•  Clemson  University,  Department  of  Psychology 

•  ENIT,  France 

•  George  Mason  University 

•  Georgia  Tech,  Modeling  and  Simulation  Research  and  Education  Center  (MSREC) 

•  Institut  National  de  Recherche  en  Informatique  et  en  Automatique,  France  (INRIA) 

•  Miami  University 

•  MIT  Lincoln  Laboratories 

•  MIT  Research  Laboratory  of  Electronics  (RLE) 

•  Old  Dominion  University,  Virginia  Modeling,  Analysis,  and  Simulation  Center  (VMASC) 

•  Queens  University,  Kingston,  Ontario 

•  University  of  California,  Berkeley,  Center  for  Design  Visualization 

•  University  of  Central  Florida,  Department  of  Industrial  Engineering 

•  University  of  Central  Florida,  Institute  for  Simulation  and  Training 

•  University  of  California,  Santa  Cruz 

•  University  of  Newcastle,  Newcastle-upon-Tyne 

•  University  of  Virginia 

•  University  of  Wisconsin 
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4.4.1.2  Corporate 

•  Bios  Group 

•  Boeing 

•  Dolby  Emergent  Designs 

•  Epic  Games 

•  John  Mason  Associates 

•  Lucasfilm  Skywalker  Sound 

•  Lucasfilm  TE1X 

•  Microstrain 

•  MITRE 

•  Potomac  Institute 

•  Science  Applications  International  Corporation  (SAIC) 

4.4.1.3  Non-Profit 

•  Center  for  Naval  Analysis 

•  Fraunhofer  Center  for  Research  in  Computer  Graphics 

•  High  Performance  Computing  Center,  Maui 

•  Institute  for  Defense  Analysis 

•  Monterey  Bay  Aquarium  Research  Institute  (MBARI) 

•  Monterey  Bay  National  Marine  Sanctuary 

•  Sea  Grant 

•  S.E.A.  Lab  Monterey  Bay 

4.4.2  Military  and  Federal 

4.4.2. 1  Air  Force 

•  Medical  Command,  San  Antonio 

4.4.2.2  Army 

•  Army  Research  Office 

•  Assistant  Sec.  Army  for  Manpower  &  Reserve  Affairs 

•  Office  of  Economic  &  Manpower  Assessment 

•  Sponsorship:  OEM  Analysis,  US  Army  Training  Analysis  (TRAC  Monterey),  US  Army  Training  and 
Doctrine  Command  (TRADOC),  US  Army  Operational  Test  Command,  Fort  Hood  (USAOTC) 

•  US  Army  Operational  Test  and  Evaluation  Command  (OPTEC),  Fort  Hood 

•  Sponsorship:  High-resolution  database  creation 
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4A.2.3  Marine  Corps 

•  Marine  Corps  Combat  Development  Command  (MCCDC) 

•  Sponsorship:  Adaptive  Exploration  of  Agent-Based  Command  and  Control  Simulations 

•  Marine  Corps  Combat  Development  Center  Training  and  Education  Command  (MCCDC  TECOM) 

•  Sponsorship:  Scenario  authoring  and  visualization  for  advanced  graphical  environments  (SAVAGE) 

•  Detail  Marine  Forces  Pacific 

•  Training  &  Education  Command 

4.4.2.4  Navy 

•  Chief  of  Naval  Operations,  CNO-N6,  Space  Information  Warfare  Command  and  Control  Directorate 

•  Sponsorship:  SimSecurity,  a  distance-learning  and  virtual  laboratory  for  information  assurance 

•  Commander,  Helicopter  Anti-Submarine  Wing,  Pacific  Fleet  (CHSWP) 

•  Commander,  Submarine  Development  Squadron  TWELVE 

•  HS-8  -  Helicopter  Anti-Submarine  Squadron  EIGHT,  FRS 

•  HS-10  -  Helicopter  Anti-Submarine  Squadron  TEN,  FRS 

•  Naval  Aerospace  Medical  Research  Lab 

•  Naval  Oceanographic  Office 

•  Naval  Postgraduate  School  Distance-Learning  Resource  Center 

•  Sponsorship:  Joint-Combat  Modeling  Class  (MV/OA4655) 

•  Naval  Postgraduate  School  Institutionally  Funded  Research  Program  (NIFR) 

•  Sponsorship:  Scenario  authoring  and  visualization  for  advanced  graphical  environments  (SAVAGE). 
Detail 

•  Naval  Sea  Systems  Command,  Advanced  Systems  &  Technology  Office 

•  Naval  Submarine  School 

•  Naval  Research  Laboratory 

•  Sponsorship:  Automatic  determination  of  safest  routes  for  aircraft  in  enemy  radar  environments 

•  Navy  Modeling  &  Simulation  Management  Office,  N6M 

•  Sponsorships:  NPSNET-V:  an  architecture  for  constructing  scalable,  dynamically  extensible, 
networked  virtual  environments 

•  NPSNET-V:  DBP,  vrtp  for  adaptive  XML-based  streaming  of  3D  behaviors  and  X3D 

•  Navy  Toxicology  Detachment,  Wright 

•  Patterson  Air  Force  Base 

•  Space  and  Naval  Warfare  (SPAWAR)  Systems  Center,  San  Diego 

•  Sponsorship:  Joint  Simulation  System  (JSIMS)  Marine  Corps  amphibious  operations  modeling 
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•  Naval  Undersea  Warfare  Center,  Newport  (NUWC) 

•  Sponsorship:  Scenario  authoring  and  visualization  for  advanced  graphical  environments  (SAVAGE). 
Detail 

•  Naval  Air  Warfare  Center,  Training  Systems  Division  (NAWC-TSD) 

•  Office  of  Naval  Research 

•  Sponsorship:  Immersive  audio 

•  Office  of  Science  and  Development  (OSD) 

•  Defense  Advanced  Research  Projects  Agency  (DARPA) 

•  Defense  Modeling  &  Simulation  Office  (DMSO) 

•  Sponsorship:  Scenario  authoring  and  visualization  for  advanced  graphical  environments  (SAVAGE). 
Detail 

•  Defense  Threat  Reduction  Agency  (DTRA) 

•  Sponsorships:  DIS-Java-VRML,  STRP  and  HLA/RTI  gateway  for  physically  based  battle-damage 
assessment 

•  Scenario  authoring  and  visualization  for  advanced  graphical  environments  (SAVAGE). 

•  Federal  Aeronautics  Administration 

•  Sponsorship:  VIRTE  (Virtual  Technologies  and  Environments) 

•  National  Aeronautics  and  Space  Administration  (NASA  Ames) 

•  National  Reconnaissance  Office 

•  Sponsorship:  NPSNET-V:  an  architecture  for  constructing  scalable,  dynamically  extensible, 
networked  virtual  environments 

•  National  Science  Foundation 

•  Sponsorship:  Virtual  Vaudeville 

•  Office  of  the  Director,  Operational  Test  &  Evaluation 

•  Office  of  the  Secretary  of  Defense,  Extensible-Markup  Language  Message-Text  Formats  (XML-MTF) 
working  group 

•  Sponsorship:  Generic  hub:  auto  generating  and  distributing  shared  virtual  environments  for  US  and 
Allied  operations  orders  using  XML  and  X3D 

•  US  Joint  Forces  Command  Joint  Experimentation  Directorate  (USJFCOM  J9) 

4.4.2.5  Foreign  Military 

•  Bulgarian  Military 

•  Czech  Republic  Military 
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4.5  LITERATURE  PREPARED  BY  RESEARCHERS 
4.5.1  Theses  and  Dissertations 

Arisut,  LTJG  Omer,  Turkish  Navy.  “Effects  of  Navigation  Aids  on  Human  Error  in  a  Complex  Navigation 
Task.”  MS  in  MOVES,  2002. 

Aronson,  MAJ  Warren.,  USA  “A  Cognitive  Task  Analysis  for  Close  Quarters  Battle.”  MS  in  computer 
science  in  cooperation  with  MOVES,  2002. 

Back,  LT  David,  USN.  “Agent-Based  Soldier  Behavior  in  Dynamic  3D  Virtual  Environments,”  MS  in 
MOVES,  2002. 

Brannon,  LTCOL  David,  USMC  and  Villandre,  MAJ  Michael,  USMC.  “The  Forward  Observer  Personal 
Computer  Simulator  (FOPCSIM).”  MS  in  computer  science  in  cooperation  with  MOVES,  2002. 

Calfee,  LT  Sharif,  USN.  “Autonomous  Agent-Based  Simulation  of  an  AEGIS  Cruiser  Combat  Information  Center 
Performing  Battle  Group  Air  Defense  Commander  Operations,”  MS  in  MOVES,  2003. 

Campbell,  LT  James,  USN.  “The  Effect  of  Sound  Spatialization  on  Responses  to  Overlapping  Messages,”  MS 
in  operations  research  in  cooperation  with  MOVES,  2002. 

Desypris,  LT  Georgios,  Hellenic  Navy.  “Enhancement  of  Learning  Process  in  Web-based  Courses  Using 
Combined  Media  Components,”  MS  in  computer  science  in  cooperation  with  MOVES,  2002. 

Dickie,  CAPT  Alistair,  Australian  Army.  “Modeling  Robot  Swarms  Using  Agent-based  Simulation,”  MS  in 
operations  research  in  cooperation  with  MOVES,  2002. 

Greenwald,  MAJ  Thomas  W.,  USA.  “An  Analysis  of  Auditory  Cues  for  Inclusion  in  a  Virtual  Close-Quarters 
Combat-Room  Clearing  Operation,”  MS  in  MOVES,  2002. 

Harney,  LT  James  W.,  USN.  “Analyzing  Tactical  Effectiveness  for  Anti-Terrorist  Force  Protection  (AT/FP) 
Using  X3D  Graphics  and  Agent-Based  Simulation,”  MS  in  MOVES,  2003. 

Krebs,  CDR  Eric  M.,  USNR.  “An  Audio  Architecture  Integrating  Sound  and  Live  Voice  for  Virtual 
Environments,”  MS  in  MOVES,  2002. 

Lennerton,  MAJ  Mark,  USMC.  “Exploring  a  Chromakeyed  Augmented  Virtual  Environment  as  an  Embedded 
Training  System  for  Military  Helicopters,”  MS  in  computer  science  in  cooperation  with  MOVES,  2002. 

List,  MAJ  Robert,  USMC.  “A  Rendering  System  Independent  High-Level  Architecture  Implementation  for 
Networked  Virtual  Environments,”  MS  in  computer  science  in  cooperation  with  MOVES,  2002. 

Michael,  LT  Robert,  USN  and  Staples,  LT  Zachary,  USN.  “Targeting  Networks:  Stimulating  Complex 
Adaptive  Systems  for  Accelerated  Learning  and  Organizational  Impotence,”  MS  in  MOVES,  2003. 

Mowery,  MAJ  Samuel,  USMC.  “Enhancing  the  Situational  Awareness  of  Airfield  Local  Controllers,” 
MS  operations  research  in  cooperation  with  MOVES,  2002. 

Orichel,  CAPT  Thomas,  German  Army.  “Adaptive  Rules  In  Emergent  Logistics  (ARIEL),”  MS  in  MOVES, 
2003. 
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Osbom,  CDR  Brian,  USN.  Dissertation,  “An  Agent-based  Architecture  For  Generating  Interactive  Stories,”  Ph.D.  in 
computer  science  in  cooperation  with  MOVES,  2002. 

Peitso,  LCDR  Loren,  USN.  “Visual  Field  Requirements  for  Precision  Nap-of-the-Earth  Helicopter  Flight,” 
MS  in  MOVES,  2002. 

Perkins,  MAJ  Keith  M.,  USA.  “Implementing  Realistic  Helicopter  Physics  in  3D  Game  Environments,” 
MS  in  MOVES,  2002. 

Reece,  CAPT  Jordan,  USMC.  “Virtual  Close  Quarters  Battle  (CQB)  Graphical  Decision  Trainer,”  MS  in 
computer  science  in  cooperation  with  MOVES,  2002. 

Sanders,  MAJ  Richard,  USA,  and  Scorgie,  LT  Mark,  USN.  “The  Effect  of  Sound  Delivery  Methods  on  a 
User’s  Sense  of  Presence  in  a  Virtual  Environment,”  MS  in  MOVES,  2002. 

Spears,  LT  Victor,  USN.  “Terrain  Level  of  Detail  in  First-person,  Ground-perspective  Simulation,”  MS  in 
MOVES,  2002. 

Thien,  CAPT  Robert,  USMC.  “Realistic  Airspace  Simulation  through  the  Use  of  Visual  and  Aural  Cues,” 
MS  in  computer  science  in  cooperation  with  MOVES,  2002. 

Ulate,  LT  Stephen  O.,  USN.  “The  Impact  of  Emotional  Arousal  on  Learning  in  Virtual  Environments,”  MS  in 
MOVES,  2002. 

VanPutte,  MAJ  Michael,  USA.  “A  Computational  Model  and  Multi-agent  Simulation  for  Information 
Assurance,”  Ph.D.  in  computer  science  in  cooperation  with  MOVES,  2002. 

Wu,  LT  Hsin-Fu,  USN.  “Spectral  Analysis  and  Sonification  of  Simulation  Data  Generated  in  a  Frequency 
Domain  Experiment,”  MS  in  operations  research  in  cooperation  with  MOVES,  2002. 

4.5.2  Conferences:  Accepted/Published  Papers 

Andrade,  S.,  Rowe,  N.,  Gaver,  D.,  and  Jacobs,  P.  “Analysis  of  Shipboard  Firefighting-team  Efficiency  Using 
Intelligent-agent  Simulation,”  Proceedings  of  the  2002  Command  and  Control  Research  and  Technology 
Symposium,  Naval  Postgraduate  School,  Monterey,  CA,  June  11-13,  2002. 

Barkdoll,  T.C.,  Gaver,  D.P.,  Glazebrook,  K.D.,  Jacobs,  P.A.,  and  Posadas,  S.  “Suppression  of  Enemy  Air 
Defenses  (SEAD)  as  an  Information  Duel,”  Naval  Research  Logistics  49\  pp.  723-742,  2002. 

Blais,  C.L.,  Brutzman,  D.,  Harney,  J.W.,  and  Weekley,  J.  “Emerging  Web-Based  3D  Graphics  for  Education 
and  Experimentation,”  Proceedings,  Interservice/Industry  Training,  Simulation,  and  Education  Conference, 
Orlando,  December  2002.  Nominated,  best  paper,  ITSEC. 

Blais,  C.,  Brutzman,  D.,  and  Harney,  Weekley,  J.  “Web-based  3D  reconstruction  of  scenarios  for  limited 
objective  experiments,”  Proceedings,  Summer  Computer  Simulation  Conference,  San  Diego,  July  2002. 

Brutzman,  D.,  Zyda,  M.,  Pullen,  M.,  and  Morse,  K.  “Extensible  Modeling  and  Simulation  Framework 
(XMSF)  Challenges  for  Web-Based  Modeling  and  Simulation,”  findings  and  recommendations  report, 
Technical  Challenges  Workshop,  Strategic  Opportunities  Symposium,  Monterey,  October  2002. 
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Gaver,  D.P.  and  Jacobs,  P.A.  “Battlespace/Information  War  (BAT/IW):  a  System-of-Systems  Model  of  a 
Strike  Operation,”  Naval  Postgraduate  School  Technical  Report,  NPS-OR-02-005,  August  2002. 

Shilling,  R.,  Zyda,  M.,  and  Wardynski,  C.  “Introducing  Emotion  into  Military  Simulation  and  Videogame 
Design:  America ’s  Army:  Operations  and  VIRTE,  “in  Proceedings  of  the  GameOn  Conference,  London, 
30  November  2002. 

Shilling,  R.D.  “Contribution  of  Professional  Sound  Design  Techniques  to  Performance  and  Presence  in 
Virtual  Environments:  Objective  Measures.”  Proceedings  of  47th  Department  of  Defense  Human  Factors 
Engineering  Technical  Advisory  Group  Meeting,  September  2002,  San  Diego,  CA. 

Shilling,  R.D.  “Enhancing  Performance  in  Tactical  Environments  Using  Immersive  Auditory  Displays  and 
Data  Sonification  Techniques.”  ONR  Cognitive  Sciences  Workshop,  George  Mason  University,  2002. 

Shilling,  R.D.  “Entertainment  Industry  Sound  Design  Techniques  to  Improve  Presence  and  Training 
Performance  in  VE,”  European  Simulation  Interoperability  Workshop,  London,  England,  2002. 

Shilling,  R.D.,  Zyda,  M.,  and  Wardynski,  E.  “Introducing  Emotion  into  Military  Simulation  and  Videogame 
Design:  America ’s  Army:  Operations  and  VIRTE,”  European  Simulation  Office,  Game-On  2002,  London, 
England,  2002. 

Trefftz,  H.,  Marsic,  I.,  and  Zyda,  M.  “Handling  Heterogeneity  in  Networked  Virtual  Environments,” 
Proceedings  of  IEEE  VR,  Orlando,  Florida,  25-27  March  2002. 

Trefftz,  H.,  Marsic,  I.,  and  Zyda,  M.  “Handling  Heterogeneity  in  Networked  Virtual  Environments,” 
Presence,  Vol.  12,  No.  1,  February  2003:  pp.  38-52,  (revised  from  IEEE  VR  2002  paper). 

VanPutte,  M.,  Osborn,  B.,  and  Hiles,  J.  “A  Composite  Agent  Architecture  for  Multi-Agent  Simulations,” 
1 1th  Computer  Generated  Forces  and  Behavioral  Representation  Conference,  Orlando,  FL,  May  2002. 

4.5.3  Invited  Papers 

Stanney,  K.M.  and  Zyda,  M.  “Virtual  Environments  in  the  21st  Century,”  in  Handbook  of  Virtual 
Environments  -  Design,  Implementation,  and  Applications,  Lawrence  Erlbaum  Associates,  Publishers, 
Mahwah,  NJ,  2002. 

Zyda,  M.,  Mayberry,  A.,  Wardynski,  C.,  Shilling,  R.,  and  Davis,  M.  “The  MOVES  Institute’s  America’s 
Army:  Operations  Game,”  Proceedings  of  ACM  SIGGRAPH  2003  Symposium  on  Interactive  3D  Graphics, 
28-30  April  2003:  217-218,  color  plate,  p.  252. 

Zyda,  M.  and  Bennett,  D.  “The  Last  Teacher,”  in  2020  Visions,  from  the  Summit  and  Press  Conference  on 
the  Use  of  Advanced  Technologies  in  Education  and  Training,  US  Department  of  Commerce,  17  and 
27  September  2002. 

Zyda,  M.,  Hiles,  J.,  Mayberry,  A.,  Wardynski,  C.,  Capps,  M.,  Osborn,  B.,  Shilling,  R.,  Robaszewski,  M.,  and 
Davis,  M.  “Entertainment  R&D  for  Defense,”  IEEE  Computer  Graphics  and  Applications,  January/February 
2003. 
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4.5.4  Books,  Chapters 

Cockayne,  W.  and  Darken,  R.P.  (in  press).  “The  Application  of  Human  Ability  Requirements  to  Virtual 
Environment  Interface  Design  and  Evaluation.”  Handbook  of  Task  Analysis  for  Human-Computer  Interaction. 
Diaper,  D.  &  Stanton,  N.  Eds. 

Shilling,  R.D.  and  Shinn-Cunningham,  B.  “Virtual  Auditory  Displays.”  Virtual  Environments  Handbook, 
Kaye  Stanney,  New  York,  Erlbaum,  2003. 

4.5.5  Videotape,  Live  Demos,  and  Press 

Lenoir,  Tim.  “Fashioning  the  Military-Entertainment  Complex,”  in  Correspondence,  an  International  Review 
of  Culture  &  Society,  Publisher:  Council  on  Foreign  Relations,  Issue  No.  10,  Winter  2002-2003. 

17  September  and  27  September  2002:  Summit  &  Press  Conference  on  the  Use  of  Advanced  Technologies  in 
Education  and  Training,  US  Department  of  Commerce,  “2020  Visions.” 

20  August  2002:  Monterey  County  Herald,  “Navy  School  Computer  Program  is  Virtual  Hit,”  by  Kevin  Howe. 
16  April  2002:  ABC  7  News,  KGO,  “Hope  Against  Terrorism,”  Katarina  Rusk. 

4.5.6  E3  Coverage  of  the  Announcement  of  AA 

21  May  2002 

Kuroshin,  “AA:  The  Game,”  by  AmberEyes. 

USA  Today,  “Army  gives  new  meaning  to  war  games  -  on  a  PC,”  by  Marc  Saltzman. 

22  May  2002 

US  Army  official  press  release. 

LA  Times  front  page  on  opening  day  of  E3  (only  E3  Story  printed),  “Army’s  New  Message  to  Young 
Recruits:  Uncle  Sim  Wants  You,”  by  Alex  Pham. 

LA  Times  movie  clip  from  Web  site,  with  Alex  Pham. 

Army  Link  News,”  Army  Game  to  Debut  This  Summer,”  by  Heike  Hasenauer. 

BusinessWeek  Online,  “The  Army’s  New  Killer  App,”  by  Arlene  Weintraub. 

Chicago  Tribune,  “Army  to  Recruits:  Uncle  Sim  Wants  You,”  by  Alex  Pham. 

CNET,  “US  Army  Invades  Game  Business,”  by  David  Becker. 

CNN.COM,  “Army  is  Looking  for  a  Few  Good  Gamers,”  by  Renay  San  Miguel. 

MSNBC.COM,  “Video  Game  Used  To  Recruit  for  Armed  Forces.” 

PC  Magazine,  “US  Army  Develops  Free  Combat  Simulation,”  by  Mark  Hachman. 

23  May  2002 

AP  photo  of  America ’s  Army  booth  at  E3. 

BBC  News,  “US  military  reveals  computer  game.” 
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Detroit  Free  Press,  “Game  Master:  Army  Game  to  Draft  Virtual  Soldiers,”  by  Jim  Schaefer. 

GameSpot,  “E3  2002:  America ’s  Army  Impressions.” 

LANParty.com,  “AA:  Operations ,”  by  Phineas. 

Mercury  News,  “US  Army  Invades  Video  Game  Territory,”  by  Anthony  Breznican. 

New  York  Times,  “Army  Recruiting  Through  Video  Games.” 

CNET  Japan,  “ America ’s  Army.” 

PC  Web,  Japan. 

Reuters,  “Army  Turns  to  Computer  Games  to  Woo  Recruits,”  by  Franklin  Paul. 

Slashdot  Reaction  to  the  E3  Announcement. 

Washington  Post,  “A  Chance  to  Be  All  You  Can  Be — in  a  Virtual  Army,”  by  Jeff  Adler. 

WFMY  news  broadcast 

Wall  Street  Journal,  “Videogame  Players  to  Get  Look  at  Life  in  the  US  Army,”  by  Patrick  Maio. 

Yahoo  News,  photos  land  2. 

24  May  2002 

Fortune,  blog  in  “Peter  Lewis  On  Technology.” 

Frictionless  Insight,  “Frictionless  Insight’s  First  Annual  E3  Awards,”  by  Kyle  Ackerman  and  Rob  de  los 
Reyes.  Best  Business  Model  awarded  to  AA. 

InformationWeek.com,  “An  Army  of  One  Online,”  by  Tischelle  George. 

Taipei  Times,  “Video-gamers  Be  All  They  Can  Be.” 

25-31  May  2002 

25  May:  UCLA  Daily  Bruin,  “Army  Targets,  Misleads  Us  Youth,”  by  Shirin  Vossoughi. 

27  May:  Frictionless  Insight,  “ America ’s  Army,  the  Official  US  Army  Game,”  Kyle  Ackerman. 

27-28  May:  User  Friendly  comic. 

29  May:  London  Times,  “War  Games,”  by  Tim  Wapshott. 

1  June  2002  and  Later  Echoes  of  E3. 

3  June:  CNN  Money,  “Your  Tax  Dollars  at  Play,”  by  Chris  Morris. 

5  June:  “Penny  Arcade”  comic  on  America ’s  Army. 

5  June:  The  Wargamer,  Best  of  E3  Awards,  Best  First  Person  Tactical  Shooter  to  America ’s  Army. 

12  June:  RealToons  comic. 

14  June:  CNET,  “Virtual  Metal  Jacket,”  by  Darren  Gladstone. 
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I  July:  Army  Memorandum,  “US  Army  Launches  America ’s  Army  PC  Game  on  the  Internet.” 

I I  July:  New  Digital  Reporter,  “E3:  America 's  Army  SimSoldier,”  by  NDR. 

1 1  July:  New  York  Times,  “Uncle  Sam  Wants  you  to  Play  This  Game,”  by  Brian  Kennedy. 

13  July:  Stars  and  Stripes,  “In  Army’s  New  Recruiting  Tool,  Shooting  The  Wrong  Guys  Can  Send  You  To  A 
Virtual  Leavenworth,”  by  Eric  B.  Pilgrim. 

23  July:  Monterey  County  Herald,  “NPS-Spawned  War  Games  Catching  on  Big,”  by  Kevin  Howe. 

1  August:  Zen  Gamer,  “ America ’s  Army  Preview,”  by  Marcin  Manek. 

5  August:  SF  Chronicle,  “The  Advertising  Game  Adopting  the  Latest  Thing  in  Advertising,  Army  Out  to  Do 
Some  Computer  Recruiting,”  by  Carrie  Kirby. 

4  October:  America ’s  Army  appears  on  the  new  TV  show  “Robbery  Homicide  Division.” 

4  October:  Salon.com,  “Weapons  of  Mass  Distraction,”  by  James  Au. 

6  October:  about.com,  “Military  Web3D/VR,”  pp.  1-2. 

14  October:  Newsweek,  “Full  Metal  Joystick,”  by  Trent  Gegax. 
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Paper  5  -  Naval  Research  Lab  (NRL)  in  VR 

Naval  Research  Lab 
4555  Overlook  Ave  SW 
Washington  DC  20375 
Telephone:  (202)  404-8624 

E-mail:  cohn@itd.nrl.navy.mil 

Key  Research  Personnel 

LT  Joseph  Cohn,  Ph.D.,  Lawrence  Rosenblum,  Ph.D.,  Roy  Stripling,  Ph.D. 


5.1  AREAS  OF  INTEREST 

There  are  three  main  Virtual  Reality  (VR)  facilities  at  the  Naval  Research  Laboratory.  The  first,  the 
Immersive  Simulation  Laboratory,  is  directed  by  Dr.  Jim  Templeman  and  focuses  on  exploring  Human 
Computer  Interaction  challenges  and  techniques  for  enhancing  immersion  (Templeman,  Denbrook  &  Sibert, 
1999).  The  second  facility,  the  Virtual  Reality  Laboratory,  is  directed  by  Dr.  Lawrence  Rosenblum  and 
focuses  on  many  of  the  technical  aspects  attendant  with  using  VR  as  well  as  Augmented  Reality.  The  third 
facility,  the  Warfighter  Human  Systems  Integration  Laboratory  (WHSIL),  is  co-directed  by  LT  Joseph  Cohn 
and  Dr.  Roy  Stripling.  The  focus  of  this  facility  is  on  developing  and  evaluating  VR  tools  for  supplementing 
training  within  the  US  Navy-Marine  Corps,  including  the  development  of  novel  methodologies  for  detecting, 
assessing  and  enhancing  performance.  These  latter  two  facilities  will  be  discussed  in  greater  detail  in  the 
following  sections. 


5.2  CURRENT  PROJECTS:  WHSIL 

5.2.1  Creating  Effective  First  Person  Training  Tools:  Evaluating  Locomotion  Interfaces 

Despite  the  potential  advantages,  several  practical  limitations  of  VR  systems  challenge  their  utility  as  training 
tools.  One  significant  limitation  is  the  method  by  which  individuals  control  their  movement  within  the  VR. 
The  optimal  control  method  for  training  purposes  would  allow  the  user  to  move  through  the  VR  by 
performing  identical  movements  in  the  real-world.  Such  a  system  would  thus  enable  the  trainee  to  experience 
first-hand  and  with  true  fidelity,  the  speed,  intensity,  and  precision  of  movements  required  to  perform  the  real- 
world  operations.  For  vehicle  based  applications,  such  as  flight  or  tank  simulators,  this  mapping  of  real  to 
virtual  interfaces  is  fairly  straightforward.  System  developers  need  only  copy  those  elements  of  the  real 
cockpit,  together  with  their  functionality,  that  are  deemed  necessary  for  training  purposes.  On  the  other  hand, 
for  non-vehicle  based  applications,  such  as  those  that  enable  individual  infantrymen  to  train  combat  operations 
in  an  urban  setting  from  a  first  person  perspective,  this  mapping  is  anything  but  straightforward.  Ignoring  the 
more  complex  question  of  how  best  to  recreate  virtually  the  many  sensory  stimuli  experienced  in  the  real 
world,  a  more  basic  question  is  simply  one  of  practicality.  In  vehicle  based  applications,  movement  is  effected 
through  a  proxy,  the  cockpit  instrumentation,  whereas  in  a  first  person  application,  movement  is  effected 
directly  through  the  individual.  While  it  is  a  relatively  simple  task  to  create  virtual,  active,  models  of  vehicle 
interfaces,  it  is  an  extremely  complex  task  to  do  so  for  the  ‘human’  interface. 
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There  are  numerous  surrogate  control  methods  which  may  address  this  problem.  The  simplest  involve  using 
keyboard  entries  or  a  high  degree  of  freedom  joystick  (i.e.,  one  capable  of  supporting  a  wide  range  of 
movements  including  body  rotation  and  translation  with  independent  control  of  head  orientation).  At  the  other 
extreme,  full-body  optical  tracking  enables  real  movements  to  be  tracked  and  translated  by  the  VR  system 
either  into  identical  movements  in  VR  (e.g.,  ducking,  peaking  around  comers,  and  head  or  body  turning), 
or  into  the  movements  that  these  ‘physical  gestures’  are  meant  to  represent  (e.g.,  walking-in-place  interpreted 
as  walking  forward  in  VR).  Intermediate  solutions  also  exist,  which,  for  example,  combine  real-world 
orientation  tracking  (through  inertial  or  optical  trackers)  with  joystick  controls  for  translation.  While 
conventional  wisdom  suggests  that  greater  fidelity  will  translate  into  enhanced  training  impact,  this  notion  is 
seldom  put  to  the  test  and  in  reality  this  likely  depends  on  the  specific  training  application  as  well. 

The  WHSIL  facility  is  conducting  a  series  of  experiments  intended  to  provide  a  fundamental  understanding  of 
the  mapping  between  control  system  ‘fidelity’  and  the  training  impact  for  tasks  common  to  CQB  in  urban 
terrain.  Rather  than  test  every  system  available,  we  have  selected  representative  systems  from  either  end  of  the 
extreme.  This  approach  was  adopted  based  on  the  assumption  that  if  no  differences  between  these  extremes 
were  found,  then  the  intermediate  solutions  would  not,  in  principle,  be  different  either.  While  it  is  likely  that 
each  control  interface  may  have  an  impact  on  the  breadth  and/or  quality  of  training,  it  is  both  necessary  and 
useful  to  evaluate  operator  performance  capabilities  with  the  designated  control  systems  in  a  simplistic  setting 
first.  If  a  given  system  is  found  to  be  unable  to  support  roughly  the  same  precision  of  user  movement,  then 
further  comparisons  become  moot. 

5.2.2  Multi-Modal  Sensory  Integration  for  Training  Transfer 

Given  that  VR  systems  supporting  urban  ops  will  be  required  to  provide  a  much  more  robust  set  of  sensory 
information,  it  is  crucial  to  develop  a  comprehensive  model  of  multimodal  integration  in  support  of  enhancing 
the  degree  to  which  VR  training  enhances  real  world  performance.  Within  such  a  framework,  one  can  start  to 
pose  research  questions  in  an  effort  to  more  fully  characterize  a  comprehensive  model  for  multi-modal 
sensory  integration  in  support  of  training  transfer.  The  ultimate  goal  is  to  explore  these  relationships  using 
conventional  training  transfer  studies,  combined  with  a  ‘sensory  knockout’  paradigm,  in  which  different  levels 
and  types  of  sensory  stimulus  are  varied,  leading  to  a  better  of  understanding  of  the  relationship  between 
multi-modal  sensory  integration  and  training  transfer. 

Two  efforts  are  currently  being  pursued.  The  first  focuses  on  Haptic  information  -  the  form  of  sensory 
stimulation  acquired  through  collisions  with  objects  (Caldwell,  2000;  Durlach  &  Mavor,  1995).  Most  VR 
training  simulations  essentially  ignore  the  haptic  domain,  relying  instead  on  the  visual  domain  and  to  a  lesser 
extent,  the  auditory  one,  to  provide  trainees  with  the  range  of  information  typically  encountered  in  the  real 
world.  This  approach  has  been  favored  in  part  due  to  limitations  of  current  technology.  Haptic  interfaces  must 
render  any  number  of  a  wide  range  of  sensations  (e.g.  force,  vibration,  texture  etc)  while  minimizing 
restrictions  on  movement,  a  synergy  of  requirements  that  has  yet  to  be  achieved.  Currently,  haptic  interfaces 
are  extremely  bulky,  have  significant  latencies,  and  fail  to  capture  the  range  of  frequencies/amplitudes 
typically  encountered  in  real  world  settings. 

Yet,  haptic  cues  play  a  critical  role  in  supporting  fully  immersive  VR  systems:  during  a  single  training 
exercise,  trainees  may  repeatedly  contact  walls,  furniture  and  other  objects  as  well  as  each  other.  Additionally, 
given  the  un-naturally  narrow  field-of-view  HMD  displays  currently  available,  haptic  information  is  even 
more  essential  for  making  the  user  aware  of  where,  when,  and  what  kind  of  contact  is  made  with  virtual 
objects  outside  of  their  limited  field-of-view.  In  the  real  world,  these  collisions  provide  trainees  with  critical 
information  that  ultimately  supports  their  mental  model  of  the  environment;  in  the  virtual  one,  which  lacks 
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these  cues,  the  mental  model  is  consequently  much  poorer.  When  supported,  training  systems  in  which  the 
haptic  information  is  coupled  with  visual  information  show  great  promise  (Merril,  2000).  When  absent,  the 
lack  of  multiple-modality  information  adversely  affects  the  degree  to  which  the  VR  training  enhances  real 
world  performance  (Cohn,  Bums,  Helmick  &  Meyers,  2000;  Birch  and  Bitterman,  1949;  Paivio,  1991).  Thus, 
there  is  great  training  potential  for  developing  a  method  for  providing  this  information  during  immersive  VR 
training. 

The  system  under  development  will  provide  haptic  information  in  a  manner  that  is  both  sensorally  meaningful 
and  in  synchrony  with  other  virtually  represented  information  modalities.  The  device  will  minimally 
encumber  the  wearer,  who  will  already  be  required  to  wear  an  HMD  and  other  devices.  It  will  fit  comfortably 
on  a  range  of  body  sizes  and  be  quick  and  easy  to  put  on  and  to  take  off.  It  will  also  be  mechanically  robust, 
operating  over  a  range  of  environmental  conditions  (temperature,  humidity,  excessive  use)  and  should  not 
require  overly-burdensome  power  supplies.  Finally,  the  system  should  provide  a  flexible  application 
programming  interface  for  device  control. 

Auditory  cues  also  play  a  pivotal  role  in  providing  trainees  with  comprehensive  training  experience 
(Greenwald,  2002).  In  order  for  such  cues  to  be  salient  when  integrated  into  a  VR  system,  they  must  be 
delivered  in  such  a  fashion  as  to  preserve  both  the  spatial  and  temporal  qualities  of  the  ‘real’  cues  (Brockhurst, 
1995).  Since  the  information  extracted  by  the  human  auditory  system  is  dependent  on  the  structure  of  the 
individual’s  receiving  organ,  models  supporting  the  transmission  of  these  stimuli  are  typically  developed 
based  on  individual  Head  Related  Transfer  Functions  (HRTFs)  (Kistler  and  Wightman,  1992).  Yet,  this  is 
often  a  laborious  exercise,  requiring  specialized  equipment  and  hours  of  data  collection  from  each  individual. 
Moreover,  a  wide  range  of  individuals  are  expected  to  utilize  these  VR  systems.  In  order  to  support  the  level 
of  independent  operation  necessitated  by  current  training  needs,  a  new  approach  is  mandated.  This  approach 
must  be  validated  both  in  terms  of  the  technology,  as  well  as  the  level  of  performance  enhancement 
attributable  to  the  inclusion  of  this  modality.  Current  research  focuses  on  developing  and  validating 
techniques  for  rendering  spatialized  audio  cues  quickly  and  effectively. 

5.2.3  Alternative  Visual  Displays 

Personnel  training  and  mission  rehearsal  are  costly,  logistically  demanding,  and  potentially  dangerous. 
VR  training  systems  hold  the  potential  to  solve  these  problems,  but  have  so  far  fallen  short  on  their  promise, 
in  part,  because  the  technologies  are  seldom  truly  low-cost  or  well  suited  for  true  deployment.  Or,  when  they 
are  designed  to  be  minimally  costly  and  maximally  deployable,  the  trade-offs  made  to  obtain  these  gains  come 
at  the  expense  of  the  types  of  training  that  they  can  support. 

High-end  VR  systems  rely  on  high-fidelity  HMDs  which  alone  can  cost  more  than  $20,000  each.  Even  at 
these  high  costs  per  unit,  however,  these  devices  offer  poor  peripheral  vision,  which  may  impair  the  training 
value  for  many  applications.  High-end  VR  systems  also  rely  upon  complex  tracking  technologies  to  maintain 
consistency  between  the  user’s  movements  and  movement  within  the  VR.  In  addition  to  high  cost  and 
fragile  calibration  procedures,  these  systems  generally  demand  a  large  footprint  for  effective  operation. 
An  alternative  high-end  approach  is  the  use  of  CAVE  technologies.  Based  on  rear-projection  systems  and 
advanced  tracking  technologies,  these  systems  eliminate  the  problem  of  limited  peripheral  vision,  however, 
they  remain  high-cost,  and  actually  increase  the  system  foot-print.  Low-end  VR  systems  can  be  as  simple  as 
desktop  displays  or  single  screen  projection  systems.  They  offer  much  lower  costs,  much  smaller  foot-prints, 
and  are  generally  easier  to  use.  However  the  trade-off  for  these  gains  is  the  lack  of  an  immersive  experience, 
which  may  impact  their  training  effectiveness  for  many  tasks  important  to  the  Navy  and  Marine  Corps. 
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A  largely  unexplored  middle  ground  is  the  use  of  small-footprint  ‘wrap-around’  video  display  systems.  Such 
systems  may  be  based  on  low-tech  solutions  such  as  multiple  front-projection  screens,  or  a  single  wrap-around 
(circular)  front  projection  screen.  These  systems  provide  fully  immersive  experiences  with  maximal  support 
for  peripheral  vision  and  require  only  one  degree  of  freedom  tracking  to  determine  the  orientation  of  the  user. 
These  systems  are  potentially  very  deployable  and  easy  to  use  as  well.  Wrap-around  video  display  systems 
may  thus  be  ideal  for  training  military  tasks  that  require  360°  situational  awareness,  such  as  dismounted  urban 
operations  and  forward  air  observers.  At  the  time  of  this  writing,  the  WHSIL  facility  has  plans  to  pursue  a 
rigorous  evaluation  of  the  training  values  of  two  different  wrap-around  video  display  systems.  Experimental 
evaluation  will  be  made  within  the  context  of  training  for  MOUT  operations  as  a  model  for  training  in 
dismounted  operational  tasks  in  general.  Final  assessment  of  training  value  will  occur  within  the  context  of  a 
real-world  training  transfer  experiment  and  will  include  an  assessment  of  system  deployability  (based  on 
system  cost,  cost  to  transport,  ease  of  use,  and  size  of  footprint). 

5.2.4  System-Independent  Measures  of  Team  Performance 

The  primary  goal  of  developing  any  training  system  is  to  provide  a  level  of  training  that  translates  to  enhanced 
performance  on  the  types  of  real  world  tasks  being  simulated.  The  principal  benefit  that  VR  systems  have  over 
real  world  training  is  that  they  offer  instructors  the  chance  to  train  situations  that  would  be  too  hazardous  or 
too  costly  to  actually  practice  in  the  real  world.  On  the  other  hand,  there  is  currently  a  lack  of  effective  team 
performance  models,  theories,  and  metrics,  which  could  predict  a  priori  how  effective  a  training  system  will 
be  before  it  is  actually  implemented,  as  well  as  inform  future  development  efforts.  Further,  there  is  a  void  of 
empirical  studies  that  demonstrate  team  training  transfer  from  VR  systems.  Given  the  increasing  reliance  on 
VR  training  systems  in  military  environments,  there  is  a  critical  need  to  identify  methodologies  for  the 
objective  measurement  and  assessment  of  individual  and  team  performance  to  ensure  that  training  systems  are 
effective  at  facilitating  the  development  and  maintenance  of  targeted  training  objectives  and  lead  to  transfer  of 
training  back  to  the  operational  environment. 

Current  measures  of  team  performance  suffer  from  a  heavy  reliance  on  discrete,  subjective  (i.e.  trained 
observer)  ratings  or  discrete  objective  outcome  measures.  Consequently,  these  measures  may  be  considered 
‘derivatives’  of  true  information  and,  as  developed,  reveal  little  about  the  dynamic  processes  through  which 

teams  respond,  evolve  and  develop.  Using  the  paradigm  of  Urban  Operations,  in  which  small  teams  ( n  —  ^  ) 
are  tasked  with  a  common  goal,  the  members  of  the  WFISIF  facility  are  pursuing  numerous  methods, 
stochastic  and  deterministic,  for  calculating  optimal  team  behavior  in  a  wide  range  of  settings,  including  those 
of  imperfect  information.  This  is  because  maximizing  team  perfonnance  is  the  key  to  quantifying  team 
performance  across  the  board,  insofar  as  the  behavior  of  any  non-optimal  team  may  be  rated  by  its 
resemblance  to  that  of  the  optimizing  team. 

In  control  theoretic  terms,  at  each  point  in  time  a  given  team  has  a  set  of  choices  available  to  it,  corresponding 
to  its  “control  variables.”  The  actual  physical  changes  that  result  from  these  control  variable  decisions 
determine  the  spatial,  time-dependent  characteristics  of  the  team,  known  as  the  “state  variables.”  Since  a 
team’s  primary  task  is  to  strategize  based  on  the  options  available  to  it  and  its  foes,  a  team’s  goal  can  be 
reinterpreted  in  this  context  as  determining  its  control  variables  optimally  as  functions  of  the  state  variables. 
We  categorize  the  wisdom,  level  of  optimality  of  a  respective  team’s  control  variable  functions  by  means  of  a 
game-theoretic  payoff  function.  In  order  for  a  team  to  be  optimal,  it  must  maximize  or,  depending  on  the 
construction,  minimize  its  payoff  function  over  the  domain  of  control,  space  variables.  In  addition,  the  theory 
assumes  initial  values  for  these  differential  equations,  i.e.,  that  at  each  point  in  time  one  has  knowledge  of  all 
of  the  state  variables  involved  and  that  he  can  thus  input  the  solutions  of  the  differential  equations  to  predict 
future  values.  While  this  is  the  case  in  situations  of  perfect  information,  in  many  military  and  other  “games,” 
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there  is  a  critical  shortage  of  knowledge  regarding  enemy  position,  firepower,  and  other  vital  state  variable 
values.  Importantly,  the  hypotheses  herein  have  wider  implications  beyond  the  improvement  of  existing 
training  regimens.  Insofar  as  they  engender  an  algorithm  for  optimal  combat  performance,  applications  of 
differential  game  theory  ought  to  play  a  pivotal  role  in  any  attempt  to  create  automaton  warriors.  Indeed 
game-theoretic  notions  have  been  utilized  in  applications  of  artificial  intelligence  to  related  fields. 


5.3  VR  LAB 

5.3.1  Mobile  Augmented  Reality 

Many  future  military  operations  will  occur  in  urban  environments.  In  principle,  many  of  the  difficulties  in 
these  environments  can  be  reduced  through  providing  individual  combatants  with  better  situation  awareness. 
The  Battlefield  Augmented  Reality  System  (BARS)  is  investigating  new  ways  of  delivering  situation 
awareness  in  real-time  to  the  individual  combatant  in  the  field.  Rather  than  hold  a  laptop  or  PDA,  the  user 
wears  a  see-through  head  mounted  display  and  a  wearable  computer.  The  position  and  orientation  of  the  user 
is  tracked  and  graphics  are  rendered  directly  in  the  display,  providing  heads-up  and  hands-free  access  to 
information.  Research  in  the  Virtual  Reality  Laboratory  (VRL)  is  focused  on  the  issues  of  developing 
effective  representations,  user  interactions  and  hybrid  tracking  techniques. 

5.3.2  Uncertainty  Visualization 

The  “Visualization  of  Battlefield  Uncertainty”  is  focused  on  visualizing  and  analyzing  the  impact  that 
environmental  uncertainty  has  on  active  acoustic  detection  schemes  used  for  ASW  target  state  prediction. 
Acoustic  transmissions,  which  are  used  for  detecting  targets,  are  subjected  to  environmental  conditions  such 
as  internal  waves,  thermal  currents,  varying  degrees  of  soil  densities  (when  bouncing  off  the  sea  bottom  and 
sea  mountains)  and  surface  scatter.  Estimates  for  the  environmental  conditions  are  translated  into 
environmental  uncertainties  such  as  transmission  loss,  reverberation  level  and  bounce  absorption.  These 
environmental  uncertainties  are  used  to  better  predict  locations  and  headings  of  targets.  The  project’s 
emphasis  is  to  develop  display  and  analysis  techniques  that  exploit  the  underlying  characteristics  of  the  multi¬ 
value,  multivariate  environmental  uncertainties  so  that  the  researchers  developing  target  state  estimations  can 
better  understand  the  impact  of  the  environmental  uncertainty  estimates.  Rich-content  and  easy-to-understand 
display  tools  were  developed  to  convey  the  environmental  uncertainty  data  to  research  statisticians.  Advanced 
visualization  hardware  including  a  4-wall  immersive  room  was  utilized  to  provide  an  analysis  environment  for 
the  researchers  to  study  the  statistics.  The  future  direction  of  the  project  is  to  extend  the  tools  and  develop 
capabilities  for  appropriate  end-users  (e.g.,  sonar  operators)  that  allow  for  early  detection  and  prosecution  of 
threats. 


5.3.3  Multi-Modal  Interaction 

The  Interoperable  Multi-Modal  Interaction  and  Display  System  (IMMIDS)  system  is  developing  the 
capability  to  control  several  C2  system  interfaces  with  a  multi-modal  combination  of  the  naturalistic  input 
modalities  of  speech  and  gesture.  The  interfaces  include  (1)  a  2D  map-based  interface,  where  the  user  gestures 
with  a  2D  digital  pen  on  a  tablet  computer,  (2)  a  3D  map-based  virtual  reality  (VR)  interface,  where  the  user 
gestures  with  a  6  degree-of-freedom  tracked  flightstick,  and  (3)  a  mobile,  wearable  augmented  reality  (AR) 
system,  where  the  user  gestures  using  head  orientation,  eye  gaze  orientation,  and  arm  pointing  gestures. 
Typical  examples  of  the  types  of  commands  which  the  system  can  recognize  include:  (1)  “phase  line  red 
<indicate  line  with  ink,  or  outdoors  with  pointing  gesture>”,  (2)  “there  is  a  sniper  in  this  building  <indicate 
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building  with  ink  mark,  or  outdoors  by  pointing>”,  “mine  field  <indicate  area  with  ink,  or  pointing  gesture>”. 
Interacting  in  a  naturalistic,  multi-modal  manner  will  yield  a  number  of  advantages.  First,  naturalistic  input 
modalities  bring  the  level  of  abstraction  of  the  human-machine  discourse  closer  to  that  employed  when 
humans  talk  to  each  other.  This  can  reduce  the  training  necessary  to  use  the  system,  and  it  can  facilitate 
pervasively-aware  interfaces,  where  the  system  tracks  human-human  discourse  and  provides  information 
relevant  to  the  conversation.  Second,  multi-modal  I/O  has  a  number  of  advantages  over  uni-modal  I/O. 
Among  the  most  compelling  is  mutual  disambiguation;  this  is  the  property  where  one  modality  can  correct 
interpretation  errors  in  another  modality.  Studies  have  shown  that  this  property  can  reduce  recognition  errors 
by  45%.  Another  advantage  is  that  a  single  modality  can  be  employed  when  appropriate;  for  example,  gesture 
only  when  silence  is  needed,  or  voice  only  when  the  hands  are  occupied. 
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5.6  WHSIL  FACILITIES  AVAILABLE 

•  Four  networked,  high-end  immersive  systems 

•  Four  NVIS  Head  Mounted  Displays 

•  Two  tracking  systems: 

•  PhaseSpace  Optical  Tracking  Systems  (8  cameras  per  system) 

•  Eight  InterSense  InterCube2  inertial  trackers  (2  Cubes  per  system) 

•  One  Single  Screen  projection  system 

•  HP  LCD  projector 

•  Hit  Detection  System 

•  Numerous  PCs 

•  Two  Kaiser  Head  Mounted  Displays 

5.7  VR  LAB  R&D  LABORATORY  FACILITIES  AVAILABLE 

•  Immersive  Room  (CAVE) 

•  Responsive  Workbench 

•  ORAD  DVG  8  nodes  Cluster 

•  SGI  ONYX  Workstations 

•  Numerous  multiprocessor  PCs 

•  Trackers: 

•  Ascension  Flock  of  Bird  medium  range  (magnetic) 

•  Ascension  Motion  Star  (wireless) 

•  Polhemus  Magnetic  Trackers 

•  InterSense  IS900  Trackers  (ultrasonic  -  inertial) 

•  Dynasight  Optical  Tracker 

•  6  DOF  Motion  Stage  0.01  mm  0.1  deg  accurate 

•  Optical  See-through  displays: 

•  Stereo  Sony  Glasstron 

•  Mono  Sony  Glasstron 

•  Stereo  Video  See-through 

•  Microvision  Retinal-scanning  display 
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•  CyberGrasp/Cyberforce  hand  and  arm  force  feedback  device 

•  Phantom  Haptics  Display 

•  Plasma  Screens 

•  Quantum  3D  Termite  Wearable  Computers 

•  A  variety  of  Wearable  computers  /  equipment  /  modules 

•  Ashtech  Differential  Kinematic  GPS  (3DOF  position) 

•  ADU5  6DOF  GPS  tracker  (6DOF  position  and  orientation) 

•  Video  capture  and  editing  hardware 
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Paper  6  -  U.S.  Army  Research,  Development,  &  Command  (RDECOM) 
Simulation  &  Training  Technology  Center  (STTC) 

12350  Research  Pkwy 
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Fax:  (407)  384-5440 

E-mail:  brian.comer@us.army.mil 


Key  Research  Personnel 

James  Grosse,  Brian  Comer,  Michelle  Mayo 


6.1  AREAS  OF  INTEREST 

Training  Simulation  Technologies  (TST)  for  Flomeland  Security  -  This  research  area  includes  the  Virtual 
Emergency  Response  Training  System  (VERTS),  Dismounted  Infantry  Semi-Automated  Forces  (DISAF), 
Individual  Combatant  Virtual  Simulation,  and  the  Massively  Multi-Player  Simulation  for  Asymmetric 
Warfare  Science  and  Technology  Objective  (STO). 


6.2  CURRENT  RESEARCH  PROJECTS 

6.2.1  Virtual  Emergency  Response  Training  System  (VERTS) 

This  program  focuses  on  developing  a  simulation  capability  to  train  First  Responder,  Civil  Support  Teams 
(CST),  and  Incident  Commanders  on  tasks  required  to  deal  with  a  weapons  of  mass  destruction  incident. 
The  VERTS  suite  consists  of  fully  immersive  and  desktop  simulators  for  live  trainees  networked  together  with 
a  semi-automated  force  application  (SAF).  The  suite  also  includes  medical  simulation  capabilities. 
CST  members  would  be  able  to  train  medical  tasks  on  a  Fluman  Patient  Simulator  (FIPS). 

6.2.2  Dismounted  Infantry  Semi- Automated  Forces  (DISAF) 

DISAF  was  developed  to  add  Dismounted  Infantry  to  the  virtual  battlefield  in  a  realistic  fashion.  The  Infantry 
capabilities  of  simulations  such  as  SIMNET  SAF  and  ModSAF  have  been  limited  to  the  low-fidelity 
viewpoint  of  tanks.  The  primary  focus  of  DISAF  has  been  the  development  of  tactical  behaviors  for  Individual 
through  Squad  level  operations.  DISAF  is  based  on  the  OTB  SAF  architecture.  DISAF  includes  support  for 
Military  Operations  in  Urban  Terrain  (MOUT)  and  rural  terrain  operations.  Most  of  the  DISAF  behaviors  are 
based  on  validated  military  Combat  Instruction  Sets  (CISs).  A  database  development  process  was  developed 
to  generate  Compact  Terrain  Database  (CTDB)  Multiple  Elevation  Surface  (MES)  structures  from  visual 
database  files.  DISAF  supports  ‘c7’  terrain  with  MES  capabilities.  DISAF  provides  an  enhanced  2D  Plan 
View  Display  (PVD)  to  support  display  of  MES  buildings  and  new  Individual  Combatant  (IC)  icons. 

6.2.3  Individual  Combatant  Virtual  Simulation 

This  research  initiative  focuses  on  overcoming  the  critical  challenges  for  dismounted  soldier  simulation  by 
building  on  the  previous  efforts  in  the  development  and  use  of  virtual  simulations.  RDECOM  is  currently 
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working  to  enhance  the  capabilities  of  the  Soldier  Visualization  Station  (SVS).  The  SVS  is  a  PC  based,  high 
fidelity,  virtual  simulation  system  developed  to  serve  as  a  training  simulator  for  the  small  unit  leader  with  a 
focus  on  Military  Operations  in  Urban  Terrain  (MOUT).  An  acoustic  tracking  system  is  used  to  identify  the 
position/posture  of  the  immersed  soldier.  The  tracking  system  recognizes  sensors  located  on  the  weapon  and 
hat  worn  by  the  soldier.  The  SVS  operates  from  a  rear-screen  projector,  a  standard  PC,  and  low-cost  software. 
Features  include  real-time  3D  graphics,  directional  audio,  and  a  unique  user  interface  into  the  virtual 
battlefield.  The  SVS  enables  the  realistic  and  effective  integration  of  an  individual  participant  into  a 
networked  simulation. 

6.2.4  Massively  Multi-Player  Simulation  for  Asymmetrical  Warfare 

STO  OBJECTIVE  -  Conduct  research  and  generate  technology  needed  to  perform  large-scale  (massively 
multi-player  (MMP)),  persistent  (long  term),  distributed  simulation  environment  operations  in  support  of 
asymmetric  warfare  training.  Technology  will  allow  Joint  and  Army  Special  Operations  Battle  Staffs  and  the 
Psychological  Operations  Community  to  engage  and  counter  simulated  asymmetrical  and  conventional 
warfare  operations,  interactively,  in  varying  urban  settings  and  realistic  scenarios. 

The  Army  lacks  a  high-level  training  and  analysis  capability  for  long  duration,  asymmetric  missions  such  as 
multi-year  anti-terrorist  operations  in  Afghanistan  and  the  Philippines.  Current  training  opportunities  are 
focused  on  conventional  warfare  and  are  limited  to  short  duration  missions.  The  objective  of  this  research  is  to 
develop  a  large-scale,  persistent,  distributed  simulation  environment  to  train  users  for  asymmetric  threats  and 
conventional  warfare  in  a  large  theater  of  operation.  The  research  will  also  focus  on  developing  tool  sets  to 
construct  the  physical  environments  and  to  setup  the  conditions  (economy,  religions,  social  structures,  etc.) 
within  those  environments.  Weapons  of  Mass  Destruction,  terrorists’  actions,  crowd  &  hostage  situations, 
peacekeeping,  psychological  operations,  and  civil  affairs  will  be  possible  interactions  faced  by  the  users. 
Users  will  interact  with  and  against  numerous  people  in  the  environment  not  against  scripted  computer 
intelligence.  Armed  Forces  will  be  able  to  engage  in  such  simulation  environments  anytime,  anywhere  via  the 
Internet  and  other  communication  interfaces.  Transition  objective  is  to  use  the  software  as  a  potential  front- 
end  simulation  driver  to  a  WARSIM  /  OneSAF  exercise. 

6.3  RESULTS  ACHIEVED  TO  DATE 

•  Developed  the  prototype  virtual  simulation  identified  by  the  VERTS  Operational  Requirements  Document 
(ORD). 

•  Transitioned  DISAF  capabilities  to  the  One  Semi -Automated-Force  (OneSAF)  team  for  incoiporation  into 
OTB  2.0. 

•  Developed  the  prototype  Individual  Combatant  virtual  simulation  identified  by  the  Program  Manager 
Ground  Combat  Tactical  Trainer  (PM  GCTT)  as  a  likely  candidate  to  satisfy  the  requirements  of  Soldier 
CATT. 

6.4  COLLABORATIVE  PARTNERS 

•  Army  Research  Institute  (ARI) 

•  Army  Research  Laboratory  (ARL) 

•  Program  Executive  Office  Simulation  Training  and  Instrumentation  (PEO  STRI) 
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•  Fort  Benning  Dismounted  Battlespace  Battle  Lab  (DBBL) 

•  Special  Operations  Command  (SOCOM) 

•  Maneuver  Support  Center  (MAN  SCEN) 

6.5  VR  R&D  LABORATORY  FACILITIES  AVAILABLE 

•  Soldier  Visualization  Station  (Stand  up  and  workstations) 

•  Mixed  Reality  MOUT  experimental  facility 

•  Various  virtual  systems  that  rely  on  a  wearable  computer  for  image  generation 

•  Massively  Multiplayer  Environment 
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